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tion  Workforce,  the  DAU  faculty,  and  the  entire  commercial  defense 
industry.  In  2010,  the  DAUAA  Research  Paper  Competition  was  com¬ 
pleted  for  the  third  consecutive  year,  and  winners  will  be  recognized 
at  the  DAU  Acquisition  Community  Symposium  on  April  13,  2010. 
The  theme  for  research  papers  in  the  2010  competition  is  “Achieving 
Excellence  in  a  Changing  Acquisition  Environment.”  The  top  three 
papers  will  receive  the  Hirsch  Award  and  cash  prizes  of  $1,000,  $500, 
and  $250  respectively.  A  panel  of  subject  matter  experts  reviewed  all 
submitted  research  papers  and  selected  the  top  three  winners.  This 
research  paper  competition  results  from  a  special  relationship  between 
the  DAUAA,  the  DAU  Research  Department,  and  the  ARJ. 

I  am  extremely  pleased  and  proud  to  publish  the  three  winning 
papers  for  the  third  annual  DAUAA  2010  Research  Paper  Competition 
in  this  issue  of  the  ARJ,  along  with  four  other  outstanding  papers  that 
were  nominated  for  awards.  The  theme  for  the  2010  competition  was 
very  broad,  and  you  will  see  many  diverse  topics  in  this  issue.  A  total 
of  seven  papers  was  selected  for  publication  in  this  issue. 

The  first  place  winning  research  paper  for  the  DAUAA  2010  Re¬ 
search  Paper  Competition  is  “Acquisition  Leadership:  An  Opportunity 
Lost  for  Acquisition  Excellence?”  by  Michael  J.  Kotzian.  Many  scholars 
believe  that  leadership  is  one  of  the  most  important  factors— if  not  the 
most  important— that  an  organization  must  possess  to  be  successful 
in  today’s  global  environment.  The  defense  acquisition  management 
system  has  continued  to  be  questioned  in  terms  of  delivering  cost- 
effective  and  timely  complex  weapons  systems  that  meet  warfighter 
requirements.  As  a  means  to  improve  this  ongoing  dilemma,  this  pa¬ 
per  posits  that  increased  attention  across  the  acquisition  community 
should  be  paid,  not  to  improved  processes  and  procedures,  but  to 
leadership.  The  importance  of  leadership  competency  is  reviewed 
from  the  perspective  of  the  Service’s  Professional  Military  Education 
institutions.  As  the  organization  responsible  for  training  the  acquisi¬ 
tion,  technology,  and  logistics  workforce  on  the  complexities  of  the 
defense  acquisition  management  system,  the  DAU  is  called  upon  to 


Welcome  to  this  very  special  commemo¬ 
rative  issue  of  the  Defense  Acquisition  Review 
Journal  (ARJ).  In  2008,  the  Defense  Acquisition 
University  Alumni  Association  (DAUAA),  along 
with  the  DAU  Research  Department,  initiated 
the  annual  DAUAA  2010  Research  Paper  Com¬ 
petition  for  the  DoD  acquisition  community, 
including  all  members  of  the  Defense  Acquisi- 


improve  its  entry  and  mid-level  course  offerings  associated  with  ac¬ 
quisition  by  providing  meaningful  instruction  regarding  foundations  of 
leadership  competency. 

The  second  place  winning  paper  is  “The  Product  Support  Manager: 
Achieving  Success  in  Executing  Life  Cycle  Management  Responsibili¬ 
ties,”  by  Bill  Kobren.  In  October  2009,  President  Obama  signed  the 
Fiscal  Year  2010  National  Defense  Authorization  Act  (Public  Law  111- 
84).  The  legislation  contained  a  provision  in  Section  805  entitled,  "Life 
Cycle  Management  and  Product  Support,"  requiring:  (1)  the  Secretary 
of  Defense  to  issue  comprehensive  guidance  on  life-cycle  manage¬ 
ment,  and  the  development  and  implementation  of  product  support 
strategies  for  major  weapon  systems;  (2)  that  each  major  weapon  sys¬ 
tem  be  supported  by  a  product  support  manager  (PSM);  and  (3)  that 
each  PSM  position  be  performed  by  a  properly  qualified  member  of 
the  armed  forces  or  full-time  employee  of  the  Department  of  Defense. 
The  research  paper  examined  the  intent,  importance,  and  implications 
of  this  provision,  and  offered  recommendations  for  implementation 
by  the  Office  of  the  Deputy  Under  Secretary  of  Defense  for  Logistics 
&  Materiel  Readiness,  which  drafts  DoD  policy  to  implement  this  new 
statutory  requirement. 

The  third  place  winning  paper  is  “How  Well  Are  PMs  Doing?  Indus¬ 
try  View  of  Defense  Program  Manager  Counterparts,"  by  Roy  L.  Wood. 
Large,  complex  defense  acquisition  programs  have  been  plagued  by 
cost  overruns,  delayed  schedules,  and  subpar  performance.  Much  of 
the  responsibility  has  been  attributed  to  weaknesses  in  competencies 
of  government  program  managers  (PM).  This  article,  based  on  the 
author's  doctoral  dissertation,  provides  a  new  perspective  on  govern¬ 
ment  PM  competencies  by  surveying  146  defense  industry  managers 
who  work  with  the  government  PMs.  These  industry  managers  rated 
the  importance  of  common  PM  competencies  and  assessed  how  well, 
from  their  perspective,  their  government  counterparts  met  those 
competencies.  The  data  gathered  from  this  survey  revealed  several 
insights,  including  a  conclusion  that  government  PM  performance  on 
several  key  technical  skills  may  need  improvement.  The  results  of  this 
study  will  be  useful  in  assessing  training  and  development  strategies 
for  government  PMs. 

Four  additional  research  papers  were  nominated  for  publication 
in  this  commemorative  issue.  The  next  paper  is  “Improving  Defense 
Acquisition  Decision  Making”  by  COL  William  R.  Fast,  USA  (Ret.). 
This  research  investigates  evidence  and  tests  the  hypothesis  that 
the  linkages  between  the  defense  acquisition  management  system. 


the  requirements  process,  and  the  budgeting  system  are  not  suffi¬ 
ciently  defined  to  enable  the  success  of  acquisition  programs.  These 
disconnects  contribute  to  weapon  system  cost  overruns,  schedule 
delays,  and  performance  problems,  and  are  exacerbated  by  the  ever- 
changing  global  security  environment  and  rapid  pace  of  technological 
advancement.  Through  historical  research,  qualitative  and  quantitative 
analyses,  and  a  comprehensive  review  of  current  policies  and  proce¬ 
dures,  this  research  illuminates  these  areas  of  disconnect  and  proposes 
specific  recommendations  to  fix  them. 

The  fifth  research  paper  selected  for  publication  is  "It's  Time  to 
Take  the  Chill  Out  of  Cost  Containment  and  Re-Energize  a  Key  Acqui¬ 
sition  Practice,"  by  Col  Robert  L.  Tremaine,  USAF  (Ret.)  and  Donna  J. 
Seligman.  Little  will  change  regarding  the  prospect  of  future  weapons 
systems  acquisition  successes  unless  PMs  continue  to  tackle  their 
programmatic  major  hurdles  head-on,  but  one  more  than  others— cost 
containment.  Otherwise,  the  DoD  might  have  fewer  products  and 
fewer  services  to  offer  its  warfighters.  The  United  States  can  ill  afford 
any  decrease  in  its  preparedness  when  the  nation  is  currently  waging 
war  on  two  fronts.  To  better  understand  some  of  the  obstacles,  the 
authors  examined  cost  containment  in  the  context  of  Total  Life  Cycle 
Cost  Management.  They  believe  a  more  thorough  understanding  and 
aggressive  application  of  cost-containment  strategies  could  conceiv¬ 
ably  shift  acquisition  outcomes  to  a  more  cost-effective  posture. 
Responding  to  a  survey  conducted  as  part  of  this  research,  887  DoD 
acquisition  professionals  provided  input  on  cost  containment,  includ¬ 
ing  tool  types  and  associated  processes. 

The  sixth  research  paper  in  this  issue  is  “A  New  Alpha-Omega 
Map  for  Acquisition  Test  and  Evaluation,”  by  George  Axiotis.  Depart¬ 
ment  of  Defense  Acquisition  Test  and  Evaluation  (T&E)  has  been  the 
gatekeeper  to  Major  Defense  Acquisition  Program  production  since  its 
formalization  over  25  years  ago.  Yet,  the  landscape  of  the  types,  meth¬ 
ods,  and  sources  for  warfighting  systems  has  significantly  evolved. 
The  department  has  studied  and  recommended  action  for  Acquisition 
Reform  for  decades,  while  only  “tweaking  in  the  margins”  for  T&E. 
The  time  is  right  for  DoD  to  consider  a  new  approach  to  T&E,  steering 
away  from  the  “buy”  decision  to  the  more  relevant  “acceptance”  and 
“operational”  domains.  This  paper  outlines  the  issues  and  proposes 
a  new  “Alpha-Omega”  map  for  T&E  for  the  way  we  actually  procure 
DoD  systems. 

The  final  research  paper  in  this  special  issue  is  "The  F119  Engine:  A 
Success  Story  of  Human  Systems  Integration  in  Acquisition,”  by  2ndLt 
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Kevin  K.  Liu,  USMC,  Ricardo  Valerdi,  Donna  H.  Rhodes,  Col  Larry  Kimm, 
USAF,  and  Lt  Col  Alvis  Headen,  USAF.  The  Department  of  Defense 
recently  mandated  the  incorporation  of  Human  Systems  Integration 
(HSI)  early  in  the  acquisition  cycle  to  improve  system  performance 
and  reduce  ownership  cost.  However,  little  documentation  exists  of 
successful  examples  of  HSI  within  the  context  of  systems  engineer¬ 
ing,  making  it  difficult  for  the  acquisition  community  to  disseminate 
and  apply  best  practices.  This  paper  presents  a  case  study  of  a  large 
Air  Force  project  that  represents  a  successful  application  of  HSI.  The 
authors  explore  the  influence  of  both  the  Air  Force  and  the  project 
contractor.  Top-level  leadership  support  is  identified  for  integrating 
HSI  into  systems  engineering  processes  as  key  to  HSI  success,  reinforc¬ 
ing  the  importance  of  treating  HSI  as  an  Integral  part  of  pre-Milestone 
A  activities. 

And  now,  please  allow  me  a  few  parting  thoughts  ...  I  will  be  mov¬ 
ing  to  a  new  position  soon,  and  Dr.  Larrie  D.  Ferreiro  will  be  taking 
over  as  the  Executive  Editor  of  the  ARJ.  It  has  been  a  true  pleasure  to 
be  associated  with  some  of  the  finest  acquisition  professionals  from 
all  walks  of  the  DoD  and  defense  Industry,  and  to  be  able  to  share 
thoughts  and  philosophies  with  you.  My  tenure  as  Executive  Editor  of 
the  ARJ  began  in  August  2005;  since  then,  the  DAU  Press  has  worked 
with  me  to  bring  you  15  issues.  Between  teaching,  consulting,  manag¬ 
ing  individual  research  projects,  and  overseeing  the  DAU  Research 
Enterprise,  I  was  always  challenged  to  find  the  time  and  Inspiration 
to  publish  a  fresh,  quality  product ...  and  I  certainly  didn’t  do  it  alone! 
I  have  benefitted  greatly  from  the  knowledge  and  professional  judg¬ 
ment  of  many  folks  at  DAU,  and  I’d  like  to  mention  three  of  them 
specifically  by  name:  Norene  Fagan-Blanch,  managing  editor;  Collie 
Johnson,  technical  editor;  and  Ed  Boyd,  Director  of  DAU  Visual  Arts 
&  Press.  Without  their  dedication,  help,  and  advice,  I  would  not  have 
lasted  this  long.  Dr.  Ferreiro  will  be  taking  over  as  Executive  Editor 
starting  with  the  next  issue  (No.  55).  Like  me,  Larrie  comes  from  the 
Naval  side  of  the  DoD,  but  5  years  at  DAU  have  given  him  a  "purple" 
outlook.  Please  continue  for  Larrie  the  excellent  support  and  flow  of 
articles  that  I  have  enjoyed  from  many  of  you  over  the  past  5  years. 


( 


Dr.  Paul  Alfierl 
Executive  Editor 
Defense  ARJ 
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ACQUISITION  LEADERSHIP: 

AN  OPPORTUNITY 
LOST  FOR  ACQUISITION 
EXCELLENCE? 


'f  Michael  J.  Kotzian 

The  Defense  Acquisition  Management  System  (DAMS)  has 
continued  to  be  questioned  in  terms  of  delivering  cost-effective 
and  timely  complex  weapon  systems  that  meet  warfighter 
requirements.  As  a  means  to  improve  this  ongoing  dilemma, 
this  article  posits  that  increased  attention  across  the  acquisition 
community  should  be  paid,  not  to  improved  processes  and 
procedures,  but  to  leadership.  The  importance  of  leadership 
competency  is  reviewed  from  the  perspective  of  the  Services’ 
Professional  Military  Education  institutions.  As  the  organiza¬ 
tion  responsible  for  training  the  Acquisition,  Technology,  and 
Logistics  Workforce  on  the  complexities  of  the  DAMS,  the 
Defense  Acquisition  University  is  called  upon  to  improve  its 
entry-  and  mid-level  course  offerings  associated  with  acquisi¬ 
tion  by  providing  meaningful  instruction  regarding  foundations 
of  leadership  competency. 
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The  general  consensus  of  the  Department  of  Defense  (DoD)  community 
of  acquisition  practitioners  appears  to  indicate  that  the  weapons  systems 
acquisition  process  is  broken  and  needs  to  change. 

In  March  2009,  the  U.S.  Government  Accountability  Office  (GAO) 
issued  its  most  recent  annual  assessment  of  DoD  Major  Defense  Acquisition 
Programs  (MDAPs)  for  2008  by  concluding  that,  of  the  96  MDAPs  assessed, 
"total  research  and  development  costs  are  now  42  percent  higher  than 
originally  estimated,  and  the  average  delay  in  delivering  initial  capabilities 
is  now  22  months.  In  addition,  42  percent  of  the  programs  reported  a 
25  percent  or  more  increase  in  acquisition  unit  costs”  (GAO,  2009,  p. 
6).  Shortly  thereafter.  Secretary  of  Defense  Robert  Gates  held  a  news 
conference  in  April  2009  where  he  announced  major  changes  to  the  fiscal 
year  2010  defense  budget,  stating  DoD  needed  to  “reform  how  and  what 
we  buy;  meaning  a  fundamental  overhaul  of  our  approach  to  procurement, 
acquisition,  and  contracting”  (DefenseLink,  2009).  This  was  quickly  followed 
by  a  June  2009  editorial  from  Deputy  Secretary  of  Defense  William  Lynn 
III  who  wrote,  “For  the  first  time  in  decades,  the  political  and  economic 
stars  are  aligned  for  a  fundamental  overhaul  to  the  way  the  Pentagon  does 
business”  (Lynn,  2009).  Most  recently,  in  July  2009  the  Business  Executives 
for  National  Security  (BENS)  Task  Force  issued  a  report  that  identified 
end-to-end  problems  with  the  acquisition  system,  including  “requirements 
creep,  funding  instability,  poor  cost  estimating,  immature  technology,  and 
the  lack  of  flexibility  to  solve  problems.  These  are  compounded  by  the  fact 
that  many  individuals  with  little  or  no  accountability  can  profoundly  impact 
funding,  schedule,  personnel  assignments,  and  administrative  demands” 
(BENS,  2009,  p.  6). 

So  what’s  to  be  done  to  ensure  that  the  warfighter  receives  the 
most  capable  weapons  systems  that  meet  the  requirements,  while 
being  delivered  on  time  and  at  the  estimated  cost?  Regrettably,  the  vast 
majority  of  acquisition  reform  approaches  will  focus  on  adjustments  to  the 
processes  and  procedures  that  guide  the  DAMS.  As  an  example,  when  the 
guiding  document  governing  the  Defense  Acquisition  Management  System 
(DAMS)— DoD  Instruction  5000.02,  Operation  of  the  Defense  Acquisition 
System— was  recently  updated  and  released  in  December  2008  after 
months  of  anticipation,  the  result  was  a  110  percent  increase  in  content  from 
the  previous  version  (Brown,  2009).  This  was  accompanied  by  an  increase 
in  regulatory  and  statutory  documentation  requirements  in  support  of  all 
major  milestone  events. 

The  alteration  of  such  a  foundational  policy  should  be  viewed  as 
an  organizational  change.  From  a  DoD  perspective,  the  introduction  of 
increasingly  complex  and  altering  business  policies  and  procedures  can 
only  be  viewed  as  a  change  that  impacts  those  who  are  charged  with 
implementing  such  policy  and  procedures  through  enactment  of  the 
DAMS— members  of  the  Acquisition,  Technology,  and  Logistics  (AT&L) 
Workforce. 
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According  to  Linstone  and  Mitroff  (1994),  three  factors  merit 
consideration  when  implementing  change:  technical,  organizational, 
and  personal  perspectives.  Research  dealing  with  organizational  change 
unfortunately  “has  mainly  focused  on  organizational  factors”  while 
“neglecting  the  person-oriented  issues”  (Vakola,  Tsausis,  &  Nikolaou,  2004, 
p.  88).  While  people  are  the  most  important  factor  in  implementing  change, 
they  also  represent  the  most  difficult  factor  with  which  to  effect  change 
(Linstone  &  Mitroff,  1994).  For  any  organizational  change  to  be  effective, 
challenging  the  beliefs,  assumptions,  and  attitudes  of  the  workforce  is 
critical,  as  the  most  influential  leverage  point  for  meaningful  change  resides 
within  the  human  system  (Juechter,  Caroline,  &  Alford,  1998). 


Why  is  there  resistance  to  change? 
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Any  substantive  change  to  organizational  processes  will  be  viewed  by 
some  personnel  as  upsetting  and  by  others  as  cataclysmic,  which  can  lead 
to  workforce  resistance  in  accepting  the  change.  Why  is  there  resistance  to 
change?  A  growing  body  of  academic  literature  suggests  “organizational 
change  places  demands  not  only  on  the  organization,  but  also  on  the 
individual  employees,  both  physically  and  psychologically”  (Cole,  Harris,  & 
Bernerth,  2006,  p.  353).  Employees  are  now  being  asked  to  adapt  to  change 
without  disruption;  however,  resistance  to  change  is  the  more  common 
reaction  (Caldwell,  Herold,  &  Fedor,  2004).  Resistance  occurs  because  it 
threatens  the  status  quo  (Beer,  1980;  Hannan  &  Freeman,  1984;  Spector, 
1989)  or  increases  fear  and  the  anxiety  of  real  or  imagined  consequences 
(Morris  &  Raben,  1995;  Smith  &  Berg,  1987),  including  confidence  in  the 
ability  to  perform  (Morris  &  Raben,  1995;  O’Toole,  1995). 

In  a  previous  Defense  Acquisition  Review  Journai  article,  the  author 
posited  that  leadership  and  culture  were  critical  factors  when  it  came  to 
the  retention  of  DoD’s  AT&L  Workforce  (Kotzian,  2009).  This  article  will 
continue  one  of  the  threads  associated  with  the  author’s  earlier  article  in 
terms  of  pursuing  acquisition  excellence:  The  importance  of  leadership- 
posited  as  the  key  attribute  required  within  DoD’s  acquisition  community— is 
paramount  if  the  AT&L  Workforce  is  going  to  overcome  the  resistance  to 
policy  change  and  begin  to  approach,  in  some  appreciable  measure,  the 
expectations  of  taxpayers,  the  Congress,  and  most  importantly,  warfighters. 
Succinctly,  the  production  of  effective  weapons  systems  delivered  in  a 
timely  manner  at  a  reasonable  cost  will  constitute  acquisition  excellence 
and  all  it  embodies. 
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Purpose 

Many  scholars  believe  that  leadership  is  one  of  the  most  important 
factors— if  not  the  most  important— that  an  organization  must  possess 
in  order  to  be  successful  in  today’s  global  environment.  Unfortunately, 
most  of  the  readings  about  organizations  over  the  last  half-century  have 
focused  on  management.  The  management  techniques  to  be  used  in  the 
21st  century  are  expected  to  be  very  similar  to  those  used  by  management 
in  the  20th  century.  In  fact,  "similar  management  problems  have  existed  as 
long  as  human  societies  have  existed”  (Hofstede,  1999,  p.  35).  The  problem 
is  that  this  process  has  succeeded  in  developing  "generations  of  executives 
who  know  much  more  about  management  than  they  do  about  leadership” 
(Kotter,  1998,  p.  5).  In  the  absence  of  effective  leadership,  "the  probability 
that  a  firm  can  achieve  superior  or  even  satisfactory  performance  when 
confronting  the  challenges  of  the  global  economy  will  be  greatly  reduced” 
(Hitt  &  Ireland,  1999,  p.  43).  Clearly,  a  current  and  future  imperative  for 
DoD  is  to  produce  as  many  workforce  members  as  possible  who  know 
something  about  leadership  because  "people  who  are  just  managers  will 
never  produce  the  cultures  necessary  to  adapt  to  a  rapidly  changing  21st 
century”  (Kotter,  1998,  p.  5).  The  clear  observation  is  that  organizations 
“will  rise  and  fall  based  on  the  quality  of  their  leadership,”  and  this  will  be 
more  true  in  the  21st  century  as  organizations  “struggle  with  their  missions, 
identities,  and  strategies”  (Zahra,  1999,  pp.  39-40). 

When  it  comes  to  organizational  success,  the  academic  literature  is 
repetitive  and  unequivocal  in  its  advocacy  of  leadership  as  a  key  factor. 
Definitions  of  leadership  abound  as  do  academicians  who  have  studied 
leadership,  but  the  typical  gist  is  that  leadership  “involves  a  process 
whereby  intentional  influence  is  exerted  by  one  person  over  other  people 
to  guide,  structure,  and  facilitate  activities  and  relationships  in  a  group  or 
organization”  (YukI,  1998,  p.  3). 

Through  the  years,  many  different  leadership  styles  have  been  put  forth 
and  studied,  e.g..  Total  Quality  Management  or  TQM,  which  emerged  as  the 
dominant  management  style  during  the  1980s  through  1990s.  The  scientific 
study  of  leadership  can  be  roughly  divided  into  three  periods:  (1)  trait  theory, 
a  premise  that  presupposes  leaders  are  somehow  different  from  those  who 
remain  followers;  (2)  behavior  theory,  which  tries  to  search  out  behavioral 
characteristics  of  supposed  great  leaders  and  subsequently  identifies  three 
styles  on  a  continuum  from  autocratic  to  democratic  to  laissez-faire;  and  (3) 
contingency  theory,  which  suggests  that  the  success  of  any  leader  depends 
upon  applying  the  proper  methodologies  based  on  the  situation  (Chemers, 
1995).  However,  within  recent  academic  literature,  one  of  the  more  common 
themes  associated  with  organizational  success  is  that  of  transformational 
leadership  in  lieu  of  transactional  leadership  (Herold,  Fedor,  Caldwell,  & 
Liu,  2008;  Randall  &  Coakley,  2006).  Transactional  leadership  relies  on  a 
contractual  exchange  of  rewards  for  efforts  dependent  on  positive  and 
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negative  reinforcement  (Bass,  1990).  The  follower  complies  with  the  leader’s 
requests  for  the  rewards  but  is  not  necessarily  committed  to  the  leader  or 
organization  (Metscher,  2005).  A  common  view  of  transactional  leadership 
is  that  of  the  stereotypical  manager  who  simply  tasks  the  followers  and 
monitors  progress  to  ensure  the  tasks  are  correctly  accomplished. 

The  alternative  leadership  style  seemingly  better  suited  to  accomplish 
organizational  success  is  transformational  leadership  characterized  by  an 
“appeal  to  followers’  sense  of  values  ...  [and  enables  them]  to  see  a  higher 
vision  and  [encourages]  them  to  exert  themselves  in  the  service  of  achieving 
that  vision”  (Herold  et  al.,  2008,  p.  347).  This  type  of  leadership  style  relies 
more  on  personal  characteristics  such  as  charisma,  interpersonal  skills,  high 
levels  of  communication,  and  characteristics  other  than  rank  or  structure. 
Transformational  leaders  are  considered  change  agents  (Metscher,  2005) 
and,  contrary  to  transactional  leaders,  are  more  closely  associated  with  the 
term  “leader”  vice  manager. 


IF  LEADERSHIP  IS  VIEWED  AS  SUCH  AN  IMPORTANT 
FACTOR  IN  ORGANIZATIONAL  SUCCESS,  WHY 
WOULD  AN  ORGANIZATION  FAIL  TO  ENSURE  THAT 
ITS  WORKFORCE  RECEIVES  AS  MUCH  LEADERSHIP 
TRAINING  AS  EARLY  AS  POSSIBLE? 
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While  there  is  almost  universal  agreement  that  leadership  is  a  critical 
factor  to  the  success  of  any  organization— such  as  enacting  major  policy 
change  like  the  new  DoD  Instruction  5000.02— the  question  remains  as  to 
whether  leadership  is  part  of  an  individual’s  inherent  ability,  or  a  trait  that 
can  be  learned,  or  both.  The  intent  of  this  article  is  not  to  prove  any  of  these 
three  perspectives.  Rather,  this  article  examines  the  connection  between 
leadership  and  education.  If  leadership  is  viewed  as  such  an  important 
factor  in  organizational  success,  why  would  an  organization  fail  to  ensure 
that  its  workforce  receives  as  much  leadership  training  as  early  as  possible? 
Education  is  seen  as  a  “critical  component  of  managing  and  adapting  to 
change  in  any  organization  and  any  area  of  endeavor”  (Kenney,  1996,  p.  52). 

Within  DoD,  Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI) 
1800. 01D,  Officer  Professional  Military  Education  Poiicy  (OPMEP),  outlines 
“the  policies  and  procedures  necessary  to  fulfill  PME  [Professional  Military 
Education]  responsibilities”  (CJCS,  2009,  p.  2).  This  CJCSI  provides  the 
framework  for  development  of  Service  and  Joint  officers  by  organizing 
the  PME  continuum  into  five  military  educational  levels:  precommissioning, 
primary,  intermediate,  senior,  and  general/flag  officer.  It  defines  the  focus  of 
each  educational  level  in  terms  of  the  major  levels  of  war  (tactical,  operational. 
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and  strategic)  and  links  the  educational  levels  so  each  builds  upon  the 
knowledge  and  values  gained  in  previous  levels  (CJCS,  2009,  p.  A-A-1). 

In  addition,  the  CJSCI  framework  also  recognizes  both  the 
distinctiveness  and  interdependence  of  Joint  and  Service  schools  in 
officer  education.  Service  schools,  in  keeping  with  their  role  of  developing 
Service  specialists,  place  emphasis  on  education  primarily  from  a  Service 
perspective  in  accordance  with  joint  learning  areas  and  objectives  (CJCS, 
2009,  p.  A-A-1). 

Leadership  training  is  conducted  through  the  PME  institutions  of 
each  individual  Service  (Air  Force,  Army,  Navy,  and  Marine  Corps),  where 
future  leaders  "spend  their  formative  years  in  a  single  Service  culture  that 
shapes  their  attitudes,  values,  and  beliefs  about  what  constitutes  ‘good’ 
and  ‘bad’  leadership  styles”  (English,  2002,  p.  2).  Such  training  is  ultimately 
focused  on  mission  accomplishment  in  terms  of  conducting  successful 
combat  operations.  The  crucial  role  for  PME  is  to  “help  future  officers 
understand  how  the  world  is  changing  and  to  enable  them  to  determine 
how  the  military  must  change  to  fit  this  new  world”  (Kenney,  1996,  p.  53). 
The  PME  system,  according  to  Kenney,  is  uniquely  suited  to  the  vital  task  of 
preparing  future  military  leaders  not  simply  to  operate,  but  to  thrive  in  such 
an  environment,  to  adapt  to  rapidly  changing  conditions,  and  to  reorient 
their  thoughts  and  actions  in  real  time  to  contingencies  that  may  not  be 
what  they  seem  (Kenney,  1996,  p.  53). 

Alternatively,  the  mission  for  educating  DoD’s  AT&L  Workforce— those 
members  responsible  for  enacting  the  DAMS— primarily  falls  to  the  Defense 
Acquisition  University  (DAU).  This  responsibility  is  embedded  in  DAU’s 
mission  statement:  “Provide  practitioner  training,  career  management, 
and  services  to  enable  the  acquisition,  technology,  and  logistics  (AT&L) 
community  to  make  smart  business  decisions  and  deliver  timely  and 
affordable  capabilities  to  the  warfighter”  (DAU,  2008,  p.  1).  DAU  will 
perform  this  practitioner  training  through  one  or  more  pillars  comprising 
the  AT&L  Performance  Learning  Model  (PLM):  training,  continuous  learning, 
mission  assistance,  and  knowledge  sharing  (DAU,  n.d.). 

In  terms  of  training  the  AT&L  Workforce,  it  can  be  argued  that  all  PLM 
pillars  provide  opportunities  to  enhance  learning  about  the  DAMS.  For 
example,  DAU  supported  awareness  about  the  revised  DoD  Instruction 
5000.02  through  rapid-deployment  events  where  DAU  went  directly 
to  their  customer  base  to  foster  an  understanding  of  the  changes  and 
impacts  associated  with  the  new  policy  guidance.  However,  most  of  DAU’s 
mission  interface  with  the  AT&L  Workforce  in  terms  of  sheer  numbers 
is  accomplished  through  DAU  training  courses  offered  in  response  to 
enactment  of  the  Defense  Acquisition  Workforce  Improvement  Act 
(DAWIA),  which  was  initially  signed  into  law  in  1990. 

DAWIA  identifies— by  career  field  and  certification  level— the  education, 
training,  and  experience  requirements  DoD  AT&L  Workforce  members  must 
achieve  to  progress  over  time  within  DoD  (DAU,  2008).  DAU  identifies 
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the  type  of  assignment,  core  certification  standards,  and  unique  training 
positions  required  by  AT&L  Workforce  members  for  each  of  15  different 
career  fields  leading  to  Level  I,  Level  II,  and  Level  III  certification.  In  addition, 
“core  plus”  development  guidance  is  provided  for  those  AT&L  Workforce 
members  seeking  additional  guidance/knowledge  beyond  the  level 
certification  standards  (DAU,  2010). 

As  alluded  to  at  the  start  of  this  article,  the  remarks  from  DoD’s  senior 
leadership  and  recent  independent  studies  signify  that  DoD  is  undergoing 
a  significant  change  in  corporate  worldview/  as  the  organization  transitions 
from  an  industrial-age  military  to  an  information  technology-age  military, 
where  the  most  important  changes  are  projected  to  be  organizational  and 
doctrinal  (Davis,  Gompert,  Hillestad,  &  Johnson,  1998).  As  a  systematically 
entrenched  organization,  fundamental  changes  in  the  DoD’s  structure— 
indeed,  the  organization's  very  way  of  “doing  business”— will  prove  a 
daunting  task. 

The  rationale  for  the  research  undertaken  is  that,  by  any  standard, 
DoD  is  truly  a  world-wide  enterprise  spending  billions  of  dollars  on  the 
procurement  of  major  weapons  systems  intended  to  support  the  warfighter. 
To  accomplish  this  task  in  a  cost-effective  and  timely  manner,  DoD  needs 
to  ensure  that  all  AT&L  Workforce  members  responsible  for  transition 
are  properly  prepared  to  do  so.  This  article  posits  that  one  of  the  critical 
attributes  all  AT&L  Workforce  members  need— to  carry  out  this  tasking— is 
continued  training,  incorporating  the  concept  of  leadership.  Further,  such 
leadership  training  should  be  offered  as  early  as  possible  to  members  of 
the  AT&L  Workforce  who  are  seeking  DAWIA  career  field  certification 
through  DAU. 

Therefore,  the  purpose  of  this  research  article  is  to  examine  the  following 
research  question:  As  the  DoD  organization  responsible  for  educating  the 
AT&L  Workforce  on  the  DAMS,  is  DAU  missing  an  opportunity  to  provide 
leadership  training  at  the  entry-  and  mid-levels  to  the  AT&L  Workforce? 


Method 

This  article’s  research  question  involves  the  importance  of  leadership 
training  as  part  of  an  AT&L  Workforce  member’s  course  of  study  undertaken 
during  DAWIA  certification.  To  address  this  question,  this  article  relies  upon 
a  mix  of  quantitative  and  qualitative  research  methodologies  based  on 
the  notion  that  “qualitative  and  quantitative  methods  should  be  viewed  as 
complementary  rather  than  rival  camps”  (Jick,  1979,  p.  602). 
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QUANTITATIVE  METHODOLOGY 

From  a  quantitative  perspective,  this  article  references  survey  data  that 
were  collected  in  2008  as  part  of  a  study  regarding  organizational  change 
and  subcultures  (Kotzian,  2009).  The  survey  population  was  military  and 
civilian  senior  leaders,  managers,  or  professionals  associated  with  the  DoD— 
not  limited  to  the  AT&L  Workforce  but  thought  to  be  a  representative  cross- 
section,  applicable  in  general  to  the  AT&L  Workforce.  Senior  leadership 
(executive-level)  membership  was  defined  as  rank  structure  0-6  and  above 
for  military  members,  and  GS-15  (or  equivalent)  and  above  for  civilian 
members.  Mid-level  manager  (mid-level)  membership  was  defined  as  rank 
structure  0-4  and  0-5  for  military  members,  and  GS-14  and  GS-13  (or 
equivalent)  for  the  civilian  members.  Professional  (entry-level)  membership 
was  defined  as  rank  structure  0-1  through  0-3  and  noncommissioned 
officers  for  military  members,  and  GS-11  and  GS-12  for  civilian  members. 

The  survey's  sampling  frame  was  comprised  of  individuals  attending 
one  of  DoD’s  PME  academic  institutions,  which  was  meant  to  provide  a 
representative  cross-section  of  the  three  population  hierarchies  (i.e.,  senior 
leaders,  management,  and  professional)  from  which  DoD  identifies  future 
leaders,  managers,  and  professionals. 

The  chosen  survey  instrument  was  the  Organizational  Culture 
Assessment  Instrument  (OCAI),  which  is  based  on  the  Competing  Values 
Framework  (CVF).  The  CVF  was  developed  by  Quinn  and  Rohrbaugh 
(1983),  which  graphically  categorized  organizational  effectiveness 
into  four  quadrants,  separately  labeled  to  distinguish  its  most  notable 
characteristics— clan,  adhocracy,  market,  and  hierarchy.  The  clan  culture  is 
named  because  of  its  similarity  to  a  family-type  organization.  The  adhocracy 
culture  places  a  great  deal  of  emphasis  on  flexibility  and  external  focus. 
The  market  culture  refers  to  the  type  of  organization  that  is  mainly  focused 
on  external  constituencies  such  as  suppliers,  customers,  contractors, 
regulators,  etc.  The  hierarchy  culture  can  be  viewed  as  the  traditional 
bureaucracy  (Quinn  &  Rohrbaugh,  1983). 

The  survey  instrument  has  been  academically  reviewed  and  proven 
for  reliability  and  validity.  A  pretest  of  the  survey  was  conducted  with 
some  faculty  and  students  at  a  prominent  PME  academic  institution.  Slight 
word  changes  were  made  to  some  of  the  survey  questions  based  on  pre¬ 
test  feedback  to  make  the  survey  more  DoD-centric.  The  formal  survey 
instrument  was  distributed  via  electronic  mail. 

The  OCAI  uses  a  response  scale  in  which  respondents  divide  100 
possible  points  among  four  options  across  six  initial  questions.  The 
compilation  of  A  options  correlates  to  the  clan  culture;  the  compilation 
of  B  options  correlates  to  the  adhocracy  culture;  the  compilation  of  C 
options  correlates  to  the  market  culture;  and  the  compilation  of  D  options 
correlates  to  the  hierarchy  culture  (Cameron  &  Quinn,  1999).  The  summation 
of  points  within  each  quadrant  is  then  plotted  to  form  a  four-sided  profile 


Acquisition  Leadership:  An  Opportunity  Lost  for  Acquisition  Excellence? 


April  2010 


167 


that  graphically  illustrates  the  strength  of  each  culture.  Respondents  answer 
the  six  questions  two  times:  initially  to  provide  responses  regarding  how 
respondents  perceive  the  organization  as  it  currently  is  (now)  and  followed 
by  responses  as  to  how  they  would  like  to  see  the  organization  in  5  years 
(preferred).  The  applicability  of  the  survey  to  this  research  article  is  that 
one  of  the  questions  specifically  deals  with  the  topic  of  leadership. 


QUALITATIVE  METHODOLOGY 

Attempting  to  interpret  the  actions  of  humans  is  very  much  a  nonlinear 
endeavor.  Qualitative  research  is  best  used  to  understand  the  complexities 
associated  with  social  phenomena  (Tucker,  Powell,  &  Meyer,  1995)  as  it 
ensures  “a  commitment  to  seeing  the  social  world  from  the  point  of  view 
of  the  actor”  (Bryman,  1984,  p.  77). 

As  part  of  the  OCAI  survey,  an  open-ended  question  was  placed  at  the 
end  of  the  survey  tool.  Any  qualitative  responses  were  completely  voluntary 
on  the  part  of  each  respondent  and  could  address  any  aspect  that  the 
respondent  wished  to  discuss. 

Open-ended  questions  allow  researchers  to  obtain  answers  that  are 
unanticipated,  may  better  describe  the  real  views  of  the  respondents, 
and  allow  for  a  response  that  is  phrased  in  the  respondent’s  own  words 
(Fowler,  2002).  While  self-administered  open-ended  questions  may  not 
be  comparable  across  all  respondents,  the  responses  can  be  evaluated 
for  patterns  that  may  repeat  over  many  different  respondents  to  make 
generalized  observations  (Salkind,  2003). 

In  addition,  documentation  was  reviewed  for  applicability  in  support 
of  this  article’s  research  question.  Somewhat  similar  to  a  literature  review, 
this  methodology  consists  of  reviewing  documentation  “composed  and 
released  either  internally  or  for  public  consumption”  as  well  as  a  means  to 
“confirm  or  contradict  information  gathered  through  other  means”  (Salkind, 
2003,  p.  208). 


Results 

The  results  associated  with  this  article  consist  of  four  sets.  Quantitative 
results  are  provided  from  responses  received  from  the  2008  QCAI  survey 
study.  Qualitative  results  are  provided  from  documentation  reviews 
associated  with  Service  PME  curriculum,  DAU  DAWIA  curriculum,  and 
open-ended  responses  from  the  2008  QCAI  survey  study.  All  QCAI  survey 
results  referenced  as  part  of  this  article  are  related  to  survey  questions 
associated  with  the  survey’s  leadership  dimension. 
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QUANTITATIVE  ANALYSIS 

From  a  quantitative  perspective,  a  total  of  1,284  usable  OCAI  survey 
results  was  captured.  The  OCAI  survey  results  in  terms  of  cultural  values 
(clan,  adhocracy,  market,  hierarchy)  provided  a  comparison  between  the 
overall  military  and  civilian  culture  quadrants  in  terms  of  leadership  and 
are  summarized  in  Figures  1,  2,  and  3.  Figure  1  provides  a  four-sided  plot  of 
the  overall  military  sample  population  mean  averages  in  terms  of  perceived 
importance  of  the  leadership  dimension.  Figure  2  provides  a  four-sided 
plot  of  the  overall  civilian  sample  population  mean  averages  in  terms  of 
perceived  importance  of  the  leadership  dimension.  Figure  3  provides  a 
comparison  overlay  of  Figures  1  and  2  to  illustrate  commonalities  and 
differences  between  the  overall  military  and  civilian  sample  populations  in 
terms  of  perceived  importance  of  the  leadership  dimension. 

In  terms  of  the  “now”  organizational  profile  data  for  the  leadership 
dimension,  both  the  military  and  civilian  sample  populations  view  the 
market  leadership  style  as  dominant  (mean  averages  of  28.5  and  27.7, 
respectively).  In  addition,  both  the  military  and  civilian  sample  populations 
view  the  remaining  leadership  styles  in  the  same  order:  hierarchy  leadership 
style  (25.2  and  26.0,  respectively),  followed  by  the  clan  leadership  style 
(25.1  and  24.1,  respectively),  and  concluding  with  the  adhocracy  leadership 
style  (21.2  and  22.3,  respectively). 


FIGURE  1.  CULTURE  PROFILE  OF  THE  OVERALL  MILITARY 
SAMPLE  POPULATION 
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FIGURE  2.  CULTURE  PROFILE  OF  THE  OVERALL  CIVILIAN 
SAMPLE  POPULATION 
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FIGURE  3.  OVERLAY  OF  CULTURE  PROFILES-OVERALL  MILITARY 
SAMPLE  POPULATIONS 
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In  terms  of  the  “preferred”  organizational  profile  for  the  leadership 
dimension,  both  the  military  and  civilian  sample  populations  view  the  clan 
leadership  style  as  dominant  (32.7  and  32.4,  respectively).  In  addition,  both 
the  military  and  civilian  sample  populations  view  the  adhocracy  leadership 
style  as  the  next  most  desirable  (27.3  and  31.1,  respectively).  The  military 
sample  population  concludes  with  the  hierarchy  (20.4)  and  market  (19.6) 
leadership  styles,  respectively.  Meanwhile,  the  civilian  sample  population 
reverses  that  order  by  preferring  the  market  (19.1)  and  then  hierarchy  (17.3) 
leadership  styles,  respectively. 


QUANTITATIVE  ANALYSIS 

To  evaluate  the  role  of  leadership  within  the  Service  PME  curriculum, 
a  thorough  analysis  of  available  documentation  was  undertaken.  The 
importance  of  leadership  in  the  early  stages  of  Service  PME  curriculum  has 
been  a  common  theme  from  the  historical  beginnings  of  PME.  As  identified 
by  Holder  and  Murray  (1998),  leadership  was  annotated  as  a  primary 
focus  area  of  education  for  the  entry-level  military  officers  (p.  85).  Figure 
4  provides  an  overview  chart  of  the  latest  Service  PME  guidelines  (CJCS, 
2009,  p.  A-A-A-1).  According  to  this  document,  the  Chairman’s  PME  vision 
“entails  ensuring  that  officers  are  properly  prepared  for  their  leadership 
roles  at  every  level  of  activity  and  employment,  and  through  this,  ensure 
that  the  U.S.  Armed  Forces  remain  capable  of  defeating  today’s  threat 
and  tomorrow’s”  (CJCS,  2009,  p.  1).  As  early  as  the  precommissioning 
education  level,  leadership  is  one  of  the  foundational  focus  areas  (CJCS, 
2009,  p.  A-A-3). 

Leadership  development  is  prevalent  within  all  of  the  Service  PME 
institutions,  which  rely  on  similar  “frameworks”  used  to  advocate  the 
importance  of  leadership.  The  Navy  relies  upon  their  Leadership 
Competency  Model  (Department  of  the  Navy,  n.d.).  The  Marine  Corps 
promotes  14  leadership  traits  (U.S.  Marine  Corps,  n.d.).  The  Air  Force  uses 
Air  Force  Doctrine  Document  1-1  to  define  three  leadership  competencies 
from  the  tactical  through  strategic  levels  (Department  of  the  Air  Force, 
2006).  Similar  to  the  Air  Force  and  Navy,  the  Army  represents  leadership 
at  the  direct,  organizational,  and  strategic  levels  with  an  exhaustive  list 
of  competencies  (Department  of  the  Army,  2006).  The  key  to  all  of  these 
frameworks  is  that  leadership  is  a  common  thread  that  appears  early  within 
all  Service  PME  institutions. 

In  addition,  each  of  the  Service  PME  institutions  has  a  department  or 
dedicated  course  focused  on  leadership  development  and  training:  the  Air 
War  College  has  the  Department  of  Leadership  and  Ethics  (Air  War  College, 
n.d.);  the  Army  War  College  has  the  Department  of  Command,  Leadership, 
and  Management  (Army  War  College,  n.d.);  the  Naval  War  College  has  the 
College  of  Operational  and  Strategic  Leadership  (U.S.  Naval  War  College, 
n.d.);  and  the  Marine  Corps  has  dedicated  leadership  courses  as  part  of  its 


FIGURE  4.  SERVICE  OFFICER  PME  CONTINUUM 
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Command  and  Staff  College  and  Marine  Corps  War  College  (U.S.  Marine 
Corps,  n.d.).  As  a  result,  the  curriculum  for  each  of  these  Service  PME 
institutions  covers  aspects  of  leadership  as  a  focus  area  for  all  students 
starting  at  the  beginning  of  any  formal  Service  PME. 

Also,  as  referenced  earlier,  DoD  has  an  overarching  policy  governing 
officer  PME  intended  to  identify  the  “policies,  procedures,  objectives, 
and  responsibilities  for  officer  professional  military  education  (PME)  and 
joint  officer  professional  military  education  (JPME)”  (CJCS,  2009,  p.  1). 
Leadership  is  prevalent  throughout  this  overarching  guidance  document 
from  which  all  Service  PME  flows. 

From  the  DAWIA  perspective,  DAU  offers  acquisition-related  training 
for  DoD’s  AT&L  Workforce.  As  of  2008,  DAU  graduated  154,252  students: 
118,391  via  Web-based  training  and  35,861  via  resident  (face-to-face) 
training— a  300  percent  increase  since  fiscal  year  1999  (DAU,  2008,  p.  9). 
For  those  classes  required  for  AT&L  Workforce  member  Level  I,  Level  II, 
and  Level  III  certification,  100-level,  200-level,  300-level,  and  400-level 
classes  are  separated. 

DAU’s  2010  catalog  lists  a  total  of  95  classes  available  to  the  AT&L 
Workforce  in  support  of  acquisition-related  certification.  Of  these  classes, 
only  one  alludes  to  leadership:  ACQ  450,  Leading  in  the  Acquisition 
Environment.  The  ACQ  450  course  description  describes  the  class  as 
an  “action-based  learning  course”  that  “provides  an  overview  of  the 
competencies  and  skills  needed  to  lead  in  an  acquisition  environment”  (DAU, 
2010).  Targeted  attendees  for  this  class  consist  of  civilians  categorized  in 
terms  of  this  article  as  the  mid-level  (GS-13  and  up)  and  executive-level 
(0-5  and  above)  positions.  However,  Level  III  certification  is  also  identified 
as  a  course  prerequisite,  which  virtually  guarantees  that  only  the  most 
experienced  AT&L  Workforce  members  will  actually  attend  this  class. 

DAU  also  offers  tailored  400-level  classes  for  executive-level  AT&L 
Workforce  members,  primarily  personnel  assuming  the  program  manager 
role  and  associated  responsibilities  for  a  major  weapon  system  program. 
However,  these  400-level  classes  are  structured  as  “modules”  covering 
various  topics  of  interest  to  any  prospective  program  manager:  earned 
value  management,  risk  management,  acquisition  policy  and  strategy,  test 
and  evaluation,  contract  management,  financial  management,  etc.  Ten 
400-level  courses  are  offered,  but  only  two  have  a  module  with  “leadership” 
in  the  title:  The  Executive  Program  Manager’s  Course  (EPMC)  course  (PMT 
402)  has  a  module  “Leading  Change”;  and  the  Executive  Refresher  Course 
(ACQ  405)  has  a  module  “Leadership  and  Management  Projects.” 

DAU’s  most  popular  course  across  the  enterprise,  particularly  with 
entry-level  AT&L  Workforce  members,  is  ACQ  201,  Intermediate  Systems 
Acquisition,  with  an  annual  enrollment  of  about  8,000  students  (DAU,  2008). 
In  this  course,  which  includes  week-long  face-to-face  class  facilitation,  only 
one  slide  is  dedicated  to  the  topic  of  leadership.  In  addition,  the  leadership 
styles  referenced  as  part  of  this  one-slide  teaching  tool  refer  to  leadership 
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styles  that  most  academicians  would  not  typically  find  in  a  teaching 
environment.  As  with  the  majority  of  DAU’s  course  material  intended  for 
AT&L  Workforce  student  instruction,  the  specific  DAU  instructor/facilitator 
determines  the  level  of  emphasis  on  the  concept  of  leadership  and/or  the 
level  of  in-depth  discussion  regarding  the  topic  of  leadership.  Otherwise, 
the  ACQ  201  AT&L  Workforce  student  will  typically  be  exposed  to  no  more 
than  10  minutes  of  discussion  "dedicated”  to  leadership. 


Whether  A  QUALITATIVE  comment  was  positive 

OR  NEGATIVE,  THE  MESSAGE  EXPRESSED  BY 
MANY  SURVEY  RESPONDENTS  WAS  THE  CRITICAL 
IMPORTANCE  OE  LEADERSHIP. 


From  the  perspective  of  responses  to  the  survey’s  open-ended 
question,  the  majority  of  the  comments  stemmed  from  the  concept  of 
leadership  and  culture.  Whether  a  qualitative  comment  was  positive  or 
negative,  the  message  expressed  by  many  survey  respondents  was  the 
critical  importance  of  leadership.  A  common  theme  was  expressed  by  a 
military  mid-level  manager:  "The  human  element  is  the  key  to  success 
in  all  endeavors.  One  must  provide  true  leadership  to  achieve  success.” 
A  different  military  mid-level  manager  remarked,  "DoD  and  this  nation 
must  evolve  and  mature,  with  enlightened,  informed,  driven,  and  efficient 
leadership  forming  the  key  to  this  transformation.”  Another  military  mid¬ 
level  manager  said,  “Leadership  has  to  be...  able  to  take  risks  and  allow 
freedom  for  combatants  to  exercise  some  risk  taking.”  Regarding  the 
importance  of  leadership  styles,  an  executive-level  civilian  noted,  “As  the 
external  situation  changes,  the  leadership  needs  to  be  flexible  enough  to 
alter  its  ‘style’  to  the  circumstances.”  Such  remarks  were  common  across 
numerous  respondents  who  emphasized  the  importance  of  leadership  in 
DoD’s  organizational  success. 

In  addition  to  the  recognition  of  leadership’s  importance,  direct  remarks 
about  how  future  leaders  are  developed  as  part  of  a  formalized  DoD 
process— or  lack  thereof— were  provided.  One  civilian  mid-level  manager 
stated,  “The  development  of  future  leaders  through  a  systematic,  not 
flow-as-you-go  method  of  mentoring  middle-level  managers  is  the  MOST 
[respondent’s  emphasis]  important  task  an  organization  can  do  for  its 
future.”  A  more  common  remark  was  the  lack  of  future  leader  development 
such  as  the  military  mid-level  manager  who  commented  about  “investing  in 
senior  leader  development  way  too  late. ...If  you  want  to  build  better  senior 
leadership,  ...then  begin  educating  them  sooner  and  younger.”  Meanwhile, 
a  civilian  mid-level  manager  observed,  “We  need  individuals  to  step  up  and 
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take  more  leadership  roles  in  determining  what  tasks  can  be  prioritized  and 
accomplished  and  what  can  be  ignored.” 

The  failings  of  DoD  leadership  were  routinely  referenced  as  part  of  the 
survey  respondents’  feedback,  and  alluded  to  an  undercurrent  of  discontent 
regarding  quality  of  leadership  that  can  be  found  within  DoD.  One  military 
mid-level  manager  wondered  "Are  these  the  best  leaders  we  have  to  do  the 
job?”  A  civilian  mid-level  manager  noted,  “Good  mentorship,  leadership, 
and  orientation  for  newcomers  is  sorely  lacking  in  my  organization.”  One 
civilian  mid-level  manager  coined  a  new  term  to  describe  his  organizational 
leadership: 

BYOL— Bring  Your  Own  Leadership.  Our  formal  leadership  has 
been  routinely  bad.  In  the  absence  of  effective  formal  leadership, 
actual  leadership  has  become  pretty  egalitarian.  We  are  successful 
because  enough  reasonable  men  and  women  decide  that  they 
will  somehow  succeed— often  despite  rather  than  because  of— the 
formal  organization. 


Discussion 

By  virtually  any  standard,  leadership  has  been  identified  as  a  critical 
attribute  to  an  organization’s  success.  With  DoD  increasingly  viewed  as  an 
organization  that  needs  to  be  operated  from  a  business  perspective,  the 
value-added  from  exceptional  leadership  quality  in  the  development  of 
complex  weapon  systems  can  only  be  viewed  as  an  advantage. 

The  basic  premise  behind  this  article  is  that  such  value-added 
leadership  should  occur  as  early  as  possible  within  the  AT&L  Workforce.  The 
current  paradigm  could  serve  the  warfighter  or  taxpayers  in  a  more  efficient 
manner.  If  the  current  paradigm  was  working  at  peak  efficiency,  DoD  would 
not  receive  the  continual  waterfall  of  studies  and  reports  outlining  problem 
areas  with  the  DAMS  in  terms  of  cost,  schedule,  and/or  performance. 

In  analyzing  the  results  collected  for  this  article,  there  seems  to  be 
widespread  interest  in  improving  the  quality  of  leadership  throughout 
DoD,  including  within  the  AT&L  Workforce.  The  OCAI  survey  results 
shown  in  Figures  1,  2,  and  3  illustrate  that  both  DoD  military  and  civilian 
personnel  indicated  a  strong  preference  for  a  change  in  leadership  style— 
from  the  more  rigid  (market)  and  bureaucratic  (hierarchy)  approaches, 
to  approaches  that  balance  family  (clan)  and  risk-taking  (adhocracy).  The 
corollary  conclusion  to  be  drawn  is  that  the  AT&L  Workforce  is  looking  for  a 
different  leadership  approach  as  compared  to  what  is  viewed  as  the  current 
mode  of  leadership.  With  an  OCAI  survey  population  including  military 
and  civilian  members  across  all  rank  structures,  it  would  seem  a  logical 
extension  that  the  survey  results  would  translate  to  the  AT&L  Workforce  as 
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well.  So  the  question  becomes,  how  does  the  AT&L  Workforce  gain  added 
expertise  in  the  leadership  competency? 

This  article's  data  would  suggest  that  the  AT&L  Workforce  would  be  well 
served  if  DAU  closely  evaluated  each  Service’s  PME  leadership  educational 
approaches  and  applied  “best  practices”  to  the  current  acquisition-related 
curriculum.  This  conclusion  is  drawn  from  the  heavy  emphasis  of  the 
leadership  competency  at  all  levels  of  the  Services’  PME  institutions— not  just 
the  senior/executive  levels.  After  centuries  of  warfare,  the  military  profession 
has  recognized  the  criticality  of  quality  leadership  and  justly  ensured  that 
this  competency  is  advocated  from  the  very  beginning  of  a  warfighter’s 
career.  While  most  of  the  AT&L  Workforce  will  not  see  the  battlefield  during 
their  lifetime,  the  warfighters,  as  stakeholders,  are  nonetheless  related  to 
the  role  played  by  the  AT&L  Workforce— delivering  the  most  cost-effective 
and  timely  defense  weapon  systems  possible  that  meet  all  threshold  and  as 
many  objective  warfighter  requirements  as  possible. 

The  common  thread  running  through  this  article’s  data  analysis  is  that 
the  Service  PME  institutions  highlight  the  importance  of  the  leadership 
competency.  Meanwhile,  DAU  — responsible  for  educating  the  AT&L 
Workforce  on  the  functional  area  roles  and  responsibilities  associated 
with  successfully  implementing  the  DAMS— pays  much  less  attention  to  the 
leadership  competency  in  the  early  stages  of  DAWIA  certification  than  does 
the  “operational”  side  of  DoD.  Yes,  at  least  one  DAU  course  focuses  mainly 
on  leadership.  And  yes,  pockets  of  leadership  “modules”  are  associated  with 
other  DAU  courses.  But  all  of  these  points  of  leadership  instruction  from 
within  the  DAU  curriculum  are  solely  focused  at  the  senior/executive  level 
of  the  AT&L  Workforce— exactly  the  same  audience  that,  when  it  comes  to 
leadership,  possesses  the  most  career  experience  from  which  to  draw  upon. 

This  approach  used  by  DAU  to  provide  leadership  training  is  a  partial 
solution  and  a  good  start.  However,  instead  of  concentrating  all  leadership 
training  assets  at  the  most  experienced  and  senior  members  of  the  AT&L 
Workforce— arguably,  AT&L  Workforce  members  who  least  need  leadership 
training— the  workforce  would  be  better  served  if  the  DAU  approach 
to  leadership  training  would  evolve  to  one  of  increasing  leadership 
awareness  at  the  entry-  and  mid-level  segments  of  the  AT&L  Workforce. 
If  change  is  going  to  occur  in  stewardship  of  the  DAMS,  then  those  at  the 
“lower  levels”  need  to  be  fully  empowered  to  initiate  changes  and  enact 
innovative  approaches  to  better  serve  the  warfighter.  Such  empowerment 
and  innovation  can  be  enhanced  by  entry-  and  mid-level  AT&L  Workforce 
members  having  a  better  understanding  of  essential  leadership  principles. 

As  noted  earlier,  advocates  of  the  leadership  competency  note 
that  DAU’s  most  highly  attended  course  for  acquisition  professionals— 
ACQ  201B,  Intermediate  Systems  Acquisition— consists  of  a  single  slide 
discussing  leadership  qualities.  This  approach  does  not  adequately 
express  the  importance  of  strong  leadership  to  entry-  and  mid-level  AT&L 
Workforce  members  seeking  a  better  understanding  of  the  DAMS.  DAU 
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is  rightfully  proud  of  its  ability  to  glean  lessons  learned  from  a  variety  of 
sources,  and  then  apply  those  lessons  to  individual  defense  acquisition 
programs  as  a  means  of  leveraging  a  “force  multipiier”  to  the  procurement 
of  complex  weapon  systems.  DAU  now  needs  to  mirror  a  lessons-learned 
mind-set  from  the  Services’  PME  institutions  by  supplementing  the  entry- 
and  mid-level  acquisition-related  courses  offered  to  the  AT&L  Workforce 
with  a  meaningful  discussion  about  the  benefits  to  be  gained  from  the 
implementation  of  proper  leadership  principles. 


Conclusions 

Conducting  business  with  a  “status  quo”  philosophy  will  not  work  if 
meaningful  changes  are  expected  in  the  procurement  of  complex  defense 
weapon  systems.  The  creativity  and  innovation  contained  within  the  younger 
members  of  the  AT&L  Workforce— the  Generation  X’ers  and  Y’ers— offers  a 
generational  opportunity  that  may  provide  huge  dividends  in  the  battle  to 
improve  the  efficiency  of  the  DAMS.  But  this  opportunity  may  never  reach 
fruition  if  the  proper  leadership  training  is  not  incorporated  into  the  entry- 
and  mid-level  DAU  acquisition  courses.  The  various  Service  PME  institutions 
have  already  shown  that  any  improvement  to  an  organization  in  terms  of 
leadership  performance  is  partially  incumbent  upon  the  proper  exposure 
of  foundational  leadership  principles  to  the  up-and-coming  generation  of 
future  leaders  populating  the  military’s  ranks.  The  same  perspective  needs 
to  be  applied  to  the  AT&L  Workforce.  That  is,  if  any  appreciable  change 
is  expected  to  the  structure  and  implementation  of  DoD’s  DAMS,  then 
DAU  needs  to  significantly  enhance  its  treatment  of  leadership  principles 
available  at  all  levels  of  the  AT&L  Workforce  membership.  By  ignoring  the 
leadership  attribute  that  has  been  proven  to  be  a  significant  contributor 
to  any  organization’s  success,  the  entry-  and  mid-level  AT&L  Workforce 
members,  in  trying  to  enact  meaningful  change  to  implementation  of 
the  DAMS,  are  symbolically  “working  with  their  hands  tied.”  DAU  stands 
best  positioned  to  remedy  this  shortcoming  by  seriously  addressing  the 
importance  of  leadership. 
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THE  PRODUCT 
SUPPORT  MANAGER: 

ACHIEVING  SUCCESS 
IN  EXECUTING  LIFE 
CYCLE  MANAGEMENT 
RESPONSIBILITIES 

(  Bill  Kobren 

In  October  2009,  President  Barack  Obama  signed  the  Fiscal 
Year  2010  National  Defense  Authorization  Act.  The  legislation 
(Pub.  L.  m-84)  contained  a  provision  in  Section  805  entitled, 
“Life  Cycle  Management  and  Product  Support”  requiring: 

(1)  that  the  Secretary  of  Defense  issue  comprehensive  guidance 
on  Life  Cycle  Management  (LCM),  and  the  development  and 
Implementation  of  product  support  strategies  for  major  weapon 
systems;  (2)  that  each  major  weapon  system  be  supported 
by  a  Product  Support  Manager  (PSM);  and  (3)  that  each  PSM 
position  be  performed  by  a  properly  qualified  member  of 
the  armed  forces  or  full-time  employee  of  the  Department 
of  Defense.  This  article  examines  the  intent,  importance,  and 
implications  of  this  provision,  and  offers  recommendations 
for  implementation. 


Keywords:  Life  Cycle  Management  (LCM), 
Performance  Based  Logistics  (PBL),  Product 
Support  Manager  (PSM),  Product  Support 
Integrator  (PSt),  Life  Cycle  Logistics  (LCL) 
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The  Secretary  of  Defense  shall  require  that  each  major  weapon 
system  be  supported  by  a  product  support  manager...”  to  “maximize 
value  to  the  Department  of  Defense  by  providing  the  best  possible 
product  support  outcomes  at  the  lowest  operations  and  support 
cost.  (NDAA,  2009a,  p.  214) 

Supporting  and  sustaining  the  weapons  with  which  we  defend  our 
nation  is  as  old  as  and  indeed  pre-dates  the  establishment  of  this  republic. 
Yet,  as  we  move  beyond  the  first  decade  of  the  21st  century,  technology  has 
advanced  to  the  point  where  weaponry  is  not  only  expensive  to  develop, 
acquire,  operate,  sustain,  and  maintain,  but  managing  the  processes  and 
information,  and  resourcing  these  tasks  have  grown  commensurately 
in  complexity. 

While  both  the  Department  of  Defense  (DoD)  and  the  individual 
Services  each  have  long-established,  well-refined,  and  detailed  processes, 
policies,  and  procedures  in  place  to  oversee  product  support  management, 
the  processes,  titles,  responsibilities,  and  authority  vested  in  the  individual 
charged  with  ensuring  mission-ready,  available,  and  reliable  systems  have 
remained  fragmented— until  now. 

The  Fiscal  Year  2010  National  Defense  Authorization  Act  (FY2010 
NDAA)  was  signed  into  law  by  President  Barack  Obama  on  October 


FIGURE  1.  STATUTORY  LANGUAGE  CONTAINED  IN  SECTION 
805  OF  THE  FY2010  NDAA  REGARDING  THE  NEW  PRODUCT 
SUPPORT  MANAGER  (PSM) 
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(b)  Product  Support  Managers. — 

(1)  Requirement. — The  Secretary  of  Defense  shall  require 
that  each  major  weapon  system  be  supported  by  a  product 
support  manager  in  accordance  with  this  subsection. 

(2)  Responsibilities. — product  support  manager  for  a 
major  weapon  system  shall — 

(A)  develop  and  implement  a  comprehensive  product 
support  strategy  for  the  weapon  system; 

(B)  conduct  appropriate  cost  analyses  to  validate  the 
product  support  strategy,  including  cost-benefit  analyses 
as  outlined  in  Office  of  Management  and  Budget  Circular 
A-94; 

(C)  assure  achievement  of  desired  product  support  out¬ 
comes  through  development  and  implementation  of  appro¬ 
priate  product  support  arrangements; 

(D)  adjust  performance  requirements  and  resource 
allocations  across  product  support  integrators  and  product 
support  providers  as  necessary  to  optimize  implementation 
of  the  product  support  strategy; 

(E)  periodically  review  product  support  arrangements 
between  the  product  support  integrators  and  product  sup¬ 
port  providers  to  ensure  the  arrangements  are  consistent 
with  the  overall  product  support  strategy;  and 

(F)  prior  to  each  change  in  the  product  support  strategy 
or  every  five  years,  whichever  occurs  first,  revalidate  any 
business-case  analysis  performed  in  support  of  the  product 
support  strategy. 


Excerpt  from  §  805,  "Life  Cycle  Management  and  Product  Support" 
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28,  2009  (Pub.  L.  111-84).  Included  within  the  Act  is  a  significant  set  of 
legislative  provisions  that  modify  DoD  procurement  policies  and  practices, 
particularly  §  805,  “Life  Cycle  Management  and  Product  Support”  (NDAA, 
2009a)  (Figure  1). 

The  law  specifically  requires  (1)  that  the  Secretary  of  Defense  issue 
comprehensive  guidance  on  Life  Cycle  Management  (LCM),  and  the 
development  and  implementation  of  product  support  strategies  for  major 
weapons  systems;  (2)  that  each  major  weapon  system  be  supported  by 
a  Product  Support  Manager  (PSM)\  and  (3)  that  each  PSM  position  be 
performed  by  a  properly  qualified  member  of  the  armed  forces  or  full-time 
employee  of  the  DoD  (NDAA,  2009b). 

Why  Mandating  a  PSM  Matters 

Changes  legislated  by  the  FY2010  NDAA  will  usher  in  an  era  of  better 
performing  weapon  systems  for  21st  century  warfighters.  At  least  a  dozen 
important  benefits  result,  collectively  serving  as  a  basis  for  better  managing 
LCM  responsibilities.  The  new  legislation: 


FOCUSES  ON  DESIRED  PERFORMANCE  OUTCOMES 

First  and  foremost,  the  2009  legislation  reiterates  commitment 
by  Congress  and  three  presidential  administrations  to  outcome-based 
weapons  systems  support  and  sustainment  strategies— a  focus  that  dates 
back  to  and  even  prior  to  the  issuance  of  the  seminal  April  1998  Section 
912(c)  report  entitled,  "Actions  to  Accelerate  the  Movement  to  the  New 
Workforce  Vision”  (DoD,  1998).  This  report  was  published  by  then-Secretary 
of  Defense  William  Cohen  in  response  to  the  FY1998  NDAA.  The  desired 
end  state,  both  then  and  today,  is  simple  to  articulate  and  remarkably 
challenging:  to  attain  the  highest  possible  readiness  at  the  most  optimal 
cost  (A.  Estevez,  personal  communication,  November  30,  2009).  LCM  and 
formal  establishment  of  a  PSM  are  two  important  means  of  achieving  both. 


REDUCES  PRODUCT  SUPPORT  COSTS 

Weapon  system  product  support  costs  the  DoD  approximately  $132 
billion  annually  (DoD,  2009a,  p.  3),  a  considerable  sum  of  money  by 
any  measure,  and  a  significant  portion  of  the  defense  budget  otherwise 
unavailable  for  investment  elsewhere  in  research  and  development  or 
procurement  of  new  systems.  Thus,  as  the  Air  Force  so  aptly  articulates, 
“The  primary  focus  is  to  optimize  life  cycle  customer  support  and  achieve 
optimum  system  availability  at  the  lowest  total  ownership  cost.  The  life  cycle 
focus  on  weapon  system  sustainment  cost  dictates  a  seamless,  integrated, 
continuing  process  to  assess  and  improve  product  support  strategies” 
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(Department  of  the  Air  Force,  2009a).  Easier  said  than  done;  short  of 
major  technological  breakthroughs,  such  outcomes  are  generally  achieved 
through  rigorous  application  of  systems  engineering  processes,  designing 
with  supportability  in  mind,  long-term  sustainment  planning,  aggressive 
root  cause  analysis  and  failure  resolution,  proactive  obsolescence  and 
Diminishing  Manufacturing  Sources  and  Material  Shortages  (DMSMS) 
mitigation,  planned  technology  upgrades,  and  perhaps  most  important  of 
all,  a  constant  focus  on  system  Reliability,  Availability,  and  Maintainability 
(RAM).  An  unwavering  commitment  to  LCM  principles  and  practices  is 
therefore  an  essential  prerequisite  of  life-cycle  cost  containment. 


SUPPORTS  ACHIEVEMENT  OF  KEY  DEPARTMENTAL  PRIORITIES 

In  the  February  2010  Quadrennial  Defense  Review  (QDR)  Report,  the 
Department  of  Defense  reiterated  its  commitment  to  acquisition  excellence, 
life-cycle  management,  outcome-focused  sustainment,  and  public-private 
partnering  by  unequivocally  stating,  "Beyond  ensuring  that  acquisition 
efforts  begin  on  the  right  track,  the  Department  must  also  continue  to 
strengthen  the  execution  phase  of  weapons  development  programs...  [to] 
achieve  effective  life  cycle  cost  management  by  employing  readiness- 
based  sustainment  strategies,  facilitated  by  stable  and  robust  government- 
industry  partnerships”  (Department  of  Defense,  2010,  pp  78-79).  Together, 
establishment  of  the  PSM  position,  reiteration  of  DoD  commitment  to 
performance-based  sustainment  strategies,  and  establishment  of  well 
understood,  clearly  defined  PSM  roles,  responsibilities,  and  expectations 
combine  to  facilitate  fulfillment  of  the  readiness  outcomes  articulated  in 
the  2010  QDR  (Department  of  Defense,  2010). 


SUPPORTS  THE  PROGRAM  MANAGER 

“The  (law)  will  ensure  that  the  PSM  role  is  ‘clearly  designated’  within  the 
program  offices. ..bringing  the  many  roles  tied  to  life  cycle  support  under 
one  position”  (Munoz,  2009).  By  policy,  the  PM  is  the  LCM:  “The  PM  shall 
be  the  single  point  of  accountability  for  accomplishing  program  objectives 
for  total  life-cycle  systems  management,  including  sustainment.  PMs 
shall  consider  supportability,  life-cycle  costs,  performance,  and  schedule 
comparable  in  making  program  decisions”  (DoD,  2007,  p.  10).  However, 
PMs  cannot  fulfill  this  role  alone.  In  fact,  “perhaps  no  management  job 
in  DoD  is  more  demanding.  Although  the  PM  is  responsible  for  life  cycle 
support,  he  can  delegate  authority  to  the  PSM  and  now  we  have  doubled 
the  leadership  power  within  the  government  to  effectively  accomplish 
both  front-  and  back-end  attention  to  sustainment”  (R.  Fowler,  personal 
communication,  November  30,  2009).  PMs  pursue  two  primary  objectives. 
First,  the  weapons  system  should  be  designed,  maintained,  and  modified 
to  continuously  reduce  the  demand  for  logistics.  Second,  logistics  support 
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must  be  effective  and  efficient.  The  resources  required  to  provide  product 
support  must  be  minimized  while  meeting  warfighter  needs  (Defense 
Acquisition  University,  2005).  Establishment  of  a  formal  PSM  therefore 
enhances  a  PM’s  ability  to  execute  their  LCM  responsibilities  by  more  clearly 
defining  the  duties  of  a  key  staff  member. 


FACILITATES  LIFE  CYCLE  MANAGEMENT 

As  part  of  their  LCM  responsibilities,  PMs  are  charged  with  identifying, 
developing,  and  implementing  weapon  system  product  support  and 
sustainment  strategies.  Specifically,  "PMs  shall  develop  and  implement 
Performance  Based  Logistics  (PBL)  strategies  that  optimize  total  system 
availability  while  minimizing  cost  and  logistics  footprint”  (DoD,  2007, 
p.  7).  Moreover,  “life-cycle  sustainment  planning  and  execution  seamlessly 
span  a  system’s  entire  life  cycle,  from  Materiel  Solution  Analysis  to  disposal. 
It  translates  force  provider  capability  and  performance  requirements 
into  tailored  product  support  to  achieve  specified  and  evolving  life-cycle 
product  support  availability,  reliability,  and  affordability  parameters”  (DoD, 
2008a,  p.  28).  LCM  is  therefore  about  integration,  optimization,  leveraging 
capabilities,  and  achieving  readiness,  and  is  clearly  not  a  solo  endeavor.  The 
PM  requires  a  capable,  empowered,  and  well-trained  team  to  successfully 
execute  this  responsibility. 


CLEARLY  DELINEATES  INHERENTLY  GOVERNMENTAL  FUNCTIONS 

“The  statute  satisfied  congressional  concerns  that  in  some  instances 
inherently  governmental  functions  were  being  performed  by  commercial 
entities.  The  language  clearly  identifies  the  PSM  as  the  performer  of  those 
inherently  governmental  functions”  (A.  Estevez,  personal  communication, 
November  30,  2009),  enhancing  government  oversight  (Figure  2)  of 
product  support  strategy  implementation  (Figure  3). 


HELPS  ACHIEVE  LONG-TERM  BEST  VALUE  OUTCOMES 

In  formally  establishing  the  PSM,  Congress  reiterated  its  commitment  to 
weapons  systems  performance  outcomes  and  life-cycle  cost  optimization.  It 
also  made  it  abundantly  clear  that  in  PBL  arrangements,  both  government 
and  industry  entities  can  serve  as  product  support  integrators.  Thus,  a 
“clear  objective  of  both  Congress  and  the  administration  was  achieved, 
namely  to  maximize  competition,”  and  in  so  doing,  also  ensuring  “long-term 
best  value  sustainment  strategies  that  bring  a  balance  between  readiness 
and  cost”  (A.  Estevez,  personal  communication,  November  30,  2009). 
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FIGURE  2.  PRODUCT  SUPPORT  BUSINESS  MODEL 
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FIGURE  3.  WHAT  THE  PRODUCT  SUPPORT  INTEGRATOR  (PSI) 
MUST  DRIVE  AND  INTEGRATE 
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ESTABLISHES  CLEAR  LINES  OF  AUTHORITY 

Product  Support  Integration  (PSI)  is  something  industry  does  well, 
and  the  Section  805  language  allows  continued  reliance  on  industrial 
sustainment  integrators.  But  government  organizations  can  certainly  step 
up  and  become  integrators  as  well,  often  in  outcome-based  partnering 
strategies  with  industry  providers.  In  a  PBL  product  support  arrangement, 
“the  PSM  (acting  on  behalf  of  the  PM)  incorporates  the  appropriate  needs 
and  constraints  in  agreements  with  PSIs.  They,  in  turn,  ensure  that  the 
necessary  performance  requirements  to  meet  their  agreements  are  properly 
passed  to  the  lower  tier  Product  Support  Providers  (PSP),  who  accomplish 
the  product  support  activities”  (DoD,  2009b,  p.  35). 


CLEARLY  ARTICULATES  ROLES  AND  RESPONSIBILITIES 

Although  weapon  system  product  support  management  has  been 
somewhat  fragmented  in  terms  of  duty  titles,  specific  responsibilities,  and 
individual  authority  of  those  charged  to  deliver  it,  DoD  has  long  recognized 
the  importance  of  a  PSM.  "We’ve  been  doing  this  all  along;  the  intent  by 
clarifying  roles  and  responsibilities  is  to  drive  it  into  the  DNA  of  the  program 
office”  (A.  Estevez,  personal  communication,  November  30,  2009).  Indeed, 
the  Defense  Acquisition  Guidebook,  published  well  before  the  FY2010  NDAA 
was  signed,  specifically  outlines  PSM  responsibilities:  "The  day-to-day 
oversight  and  management  of  the  product  support  functions  are  typically 
delegated  to  a  product  support  manager.. .who  leads  the  development 
and  implementation  of  the  performance-based  product  support  strategy 
and  ensures  achievement  of  desired  support  outcomes.  The  product 
support  manager,  while  remaining  accountable  for  system  performance, 
can  delegate  responsibility  for  delivering  specific  outcomes.  In  doing  so,  the 
PM  and  PSM  may  employ  any  number  of.. .support  integrators  to  integrate 
support  from  all  support  sources  to  achieve  the  performance  outcomes 
specified  in  a  (PBL)  performance-based  agreement”  (DoD,  2009c, 
pp.  19-20).  Regardless,  “the  PSM  will  not  be  the  program  manager.  It  will 
probably  be  someone  with  sustainment  or  logistics  competencies  [and] 
certifications  at  a  given  level”  (Munoz,  2009)  (Figure  4). 


STANDARDIZES  TERMINOLOGY 

The  military  services  use  a  variety  of  terms  and  titles  for  the  PSM,  of 
which  the  statute  only  highlighted  a  few.  Not  only  does  this  terminology  mix 
tend  to  be  confusing,  it  risks  an  “apples  to  oranges”  comparison  of  duties, 
responsibilities,  and  authorities  granted  to  the  individuals  assigned  to  these 
positions,  particularly  in  joint  programs.  In  this  instance,  standardization 
of  terminology  will  likely  be  a  welcome,  if  not  overdue  change  for  the 
acquisition  and  sustainment  community  (Air  Force,  Army,  Navy,  and  Marine 
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FIGURE  4.  FY2010  NATIONAL  DEFENSE  AUTHORIZATION  ACT 
(NDAA),  SECTION  805  DEFINITIONS 
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(d)  Definitions. — In  this  section: 

(1)  The  term  “product  support”  means  the  package  of  sup¬ 
port  functions  required  to  field  and  maintain  the  readiness 
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and  operational  capability  of  major  weapon  systems,  sub¬ 
systems,  and  components,  including  all  functions  related  to 
weapon  system  readiness. 

(2)  The  term  “product  support  arrangement”  means  a  con¬ 
tract,  task  order,  or  any  type  of  other  contractual  arrangement, 
or  any  type  of  agreement  or  non-contractual  arrangement 
within  the  Federal  Government,  for  the  performance  of 
sustainment  or  logistics  support  required  for  major  weapon 
systems,  subsystems,  or  components.  The  term  includes 
arrangements  for  any  of  the  following: 

(A)  Performance-based  logistics. 

(B)  Sustainment  support. 

(C)  Contractor  logistics  support. 

(D)  Life-cycle  product  support. 

(E)  Weapon  systems  product  support. 

(3)  The  term  “product  support  integrator”  means  an  entity 
within  the  Federal  Government  or  outside  the  Federal  Govern¬ 
ment  charged  with  integrating  all  sources  of  product  support, 
both  private  and  public,  defined  within  the  scope  of  a  product 
support  arrangement. 

(4)  The  term  “product  support  provider”  means  an  entity 
that  provides  product  support  functions.  The  term  includes 
an  entity  within  the  Department  of  Defense,  an  entity  within 
the  private  sector,  or  a  partnership  between  such  entities. 

(5)  The  term  “major  weapon  system”  has  the  meaning 
given  that  term  in  section  2302d  of  title  10,  United  States 
Code.* 


★  Includes,  but  is  not  limited  to,  Major  Defense  Acquisition  Programs  (MDAP).  Title  10  U.S.C. 
2302d  defines  a  major  weapon  system  as  a  system  for  which  the  Department  of  Defense 
is  responsible  if  total  expenditures  for  research,  development,  test,  and  evaluation  for  the 
system  are  estimated  to  be  more  than  $115,000,000  (based  on  fiscal  year  1990  constant 
dollars);  or  the  eventual  total  expenditure  for  procurement  for  the  system  is  estimated  to  be 
more  than  $540,000,000  (based  on  fiscal  year  1990  constant  dollars). 

Because  10  U.S.C.  2430  defines  an  MDAP  as  a  DoD  acquisition  program  that  is  not  a  highly 
sensitive  classified  program  and  is  designated  as  a  major  defense  acquisition  program;  or 
that  is  estimated  to  require  an  eventual  total  expenditure  for  research,  development,  test, 
and  evaluation  of  more  than  $300,000,000  (based  on  fiscal  year  1990  constant  dollars)  or 
an  eventual  total  expenditure  for  procurement  of  more  than  $1,800,000,000  (based  on  fiscal 
year  1990  constant  dollars),  MDAPs  can  therefore  be  considered  major  weapon  systems. 
Notes: 

1:  See  also  DoD  Instruction  5000.02,  Enclosure  3,  Table  1  (DoD,  2008a)  for  related  descriptions 
and  decision  authorities  for  Acquisition  Category  (ACAT)  l-lll  programs,  including  MDAPs. 

2:  USD  AT&L  "Directive-Type  Memorandum  (DTM)  09-027  (DoD,  2009a)  revised  the  DoDI 
5000.02  definition  of  an  MDAP  to:  “a  DoD  acquisition  program  that  is  not  a  highly  sensitive 
classified  program  and  (1)  that  is  designated  by  the  USD(AT&L)  as  a  MDAP;  or  (2)  that  is 
estimated  to  require  an  eventuai  total  expenditure  for  research,  development,  test,  and 
evaluation,  INCLUDING  ALL  PLANNED  INCREMENTS,  of  more  than  $365  million  (based 
on  fiscal  year  2000  constant  dollars)  or  an  eventual  total  expenditure  for  procurement, 
INCLUDING  ALL  PLANNED  INCREMENTS,  of  more  than  $2.19  billion  (based  on  fiscal  year 
2000  constant  dollars)." 
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Corps  Representatives  to  the  DoD  Life  Cycle  Logistics  Functional  Integrated 
Process  Team  [FIPT],  personal  communications,  November  13-24,  2009; 
Department  of  the  Air  Force,  2009b).  Some  current  titles  include 
the  following: 

•  The  Air  Force  generally  uses  Director  of  Logistics  (DOL) 
terminology  in  their  acquisition  product  centers  and  System 
Sustainment  Manager  (SSM)  for  programs  in  sustainment. 

•  The  Navy  uses  terms  such  as  Director  of  Logistics,  Assistant 
Program  Manager  for  Logistics  (APML),  and  Deputy 
Assistant  Program  Manager  for  Logistics  (DAPML). 

•  The  Marine  Corps  uses  several  terms  for  logistics  leadership 
in  a  program  office,  including  Program  Management 

Team  Life  Cycle  Logistician  (PMT  LCL),  PM  Lead  LCL,  and 
Strategic  Business  Team  LCL. 

•  The  Army  uses  a  variety  of  titles,  including  Deputy  Program 
Manager  for  Logistics  (DPML),  Associate  Program  Manager 
for  Logistics  (APML),  Executive  Director  for  Logistics, 

Associate  Director  for  Logistics,  and  Logistics  Division  Chief 


ENCOURAGES  DEVELOPMENT  OF  APPROPRIATELY  RIGOROUS, 

TARGETED  TRAINING 

DoD  Life  Cycle  Logistics  (LCL)  training  provided  by  the  Defense 
Acquisition  University  (DAU),  while  robust  and  competency-based,  is  not 
specifically  tailored  for,  or  directly  targeted  at,  the  executive  level  and/or 
senior  logistician  in  a  program  office.  This  inevitably  leads  to  the  question  of 
whether  an  individual  with  Level  III  LCL  certification  has  received  sufficient 
training,  has  mastered  the  requisite  competencies,  or  has  demonstrated 
proficiencies  required  for  success  as  a  PSM. 


FURTHER  INTEGRATES  ACQUISITION  AND  SUSTAINMENT 

The  ultimate  "goal  is  to  ensure  sustainment  considerations  are 
integrated  into  all  planning,  implementation,  management,  and  oversight 
activities  associated  with  the  acquisition,  development,  production, 
fielding,  support,  and  disposal  of  a  system  across  its  life  cycle”  (DoD, 
2009c,  p.  5).  This  is  critical,  since  at  the  end  of  the  day,  the  PSM  is  the 
individual  who  will  be  tasked  to  “carry  that  ball  across  the  goal  line”  on 
behalf  of  the  PM. 
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Implications  and  Intent 

In  adopting  the  Section  805  language,  the  House  and  Senate 
conferees  were  extraordinarily  clear  in  articulating  their  intent,  specifically 
emphasizing  the  following  provisions  (NDAA,  2009b,  p.  779): 

•  “Product  support  encompasses  all  critical  functions 
related  to  weapon-system  readiness,  including  materiel 
management,  distribution,  technical  data  management, 
maintenance,  training,  cataloging,  configuration 
management,  engineering  support,  repair  parts 
management,  failure  reporting  and  analyses,  and 
reliability  growth. 

•  Included  within  logistics  and  sustainment  functions  are  the 
tasks  normally  performed  as  part  of  the  logistics  support 
required  for  a  major  weapon  system  that  are  designed  to 
focus  on  such  metrics  as  readiness,  reliability,  availability, 
mean  down  time,  customer  wait  time,  footprint  reduction, 
and  reduced  ownership  costs.... 

•  In  implementation  of  this  provision,  the  positions  of  product 
support  manager,  assistant  program  manager  for  logistics, 
deputy  program  manager  for  logistics,  and  system  support 
manager  shall  be  considered  synonymous.... 

•  The  product  support  manager  is  a  separate  position  from 
the  program  manager  with  distinct  responsibilities”  and 
“each  such  position  [shall]  be  performed  by  a  properly 
qualified  member  of  the  armed  forces  or  full-time  employee 
of  the  Department  of  Defense.” 

•  By  passing  this  language,  they  “in  no  way  intend  to  limit 
DoD  from  establishing  product  support  managers  and 
comprehensive  product  support  strategies  for  other 
acquisition  programs  that  are  not  designated  major 
weapon  systems.” 
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What  Does  This  Actually  Mean? 

Product  support,  also  referred  to  as  system  sustainment,  is  the 
package  of  support  functions  required  to  maintain  the  readiness 
and  operational  capability  of  weapon  systems,  subsystems, 
software,  and  support  systems.  (DoD,  2009b,  p.  7) 

The  nine  imperatives  covered  in  the  following  discussion  constitute  the 
author’s  view  of  what  the  FY2010  NDAA  legislation  means  to  the  life  cycle 
logistician  and  industry  counterparts: 


IT'S  ALL  ABOUT  THE  WARFIGHTER  AND  NATIONAL  SECURITY 

President  Obama,  signed  the  FY2010  NDAA  into  law  on  October  28, 
2009,  containing  language  that  stated,  "Our  defense  budget  isn't  about 
politics,  it's  about  the  security  of  our  country,  and  who  knows  that  every 
dollar  wasted  is  a  dollar  we  can't  spend  to  care  for  our  troops  or  protect  the 
homeland."  Ultimately,  the  2010  NDAA  is  all  about  supporting  the  warfighter 
and  ensuring  our  national  security— something  formal  establishment  of  a 
PSM  helps  to  achieve. 


PRODUCT  SUPPORT  APPLICATION  BROADER  THAN  LOGISTICS 

Clearly,  product  support,  while  primarily  a  logistics  and  sustainment 
function,  is  not  actually  synonymous  with  logistics.  Indeed,  product 
support  “encompasses  materiel  management,  distribution,  technical 
data  management,  maintenance,  training,  cataloging,  configuration 
management,  engineering  support,  repair  parts  management,  failure 
reporting  and  analysis,  and  reliability  growth”  (DoD,  2009b,  p.  7).  Product 
support  (and  LCM  for  that  matter)  is  therefore  truly  a  “team  sport,” 
requiring  involvement,  engagement,  resources,  expertise,  and  support  from 
across  the  acquisition  and  sustainment  domains,  including  from  program 
managers,  contracting  officers,  systems  engineers,  business  and  financial 
managers,  and  logisticians  of  varying  backgrounds,  including,  of  course, 
life  cycle  logisticians. 


ENHANCED  LIFE  CYCLE  MANAGEMENT 

LCM  is  an  essential  element  in  minimizing  life-cycle  costs  and 
maximizing  weapon  system  performance  and  availability,  a  point  repeatedly 
emphasized  in  DoD  acquisition  guidance.  Establishing  a  PSM  to  assist 
in  carrying  this  out  enhances  prospects  for  successfully  achieving  true 
LCM  outcomes,  while  providing  for  greater  flexibility  in  determining  long¬ 
term  product  support  and  sustainment  resourcing  requirements,  and 
establishment  of  subsequent  resourcing  decisions.  “Maintaining  flexibility 
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for  long-term  product  support  strategies  is  a  key  ingredient  of  the  new 
statute”  (R.  Fowler,  personal  communication,  November  30,  2009). 


APPLICABILITY  TO  MAJOR  WEAPON  SYSTEMS 

Words  matter.  The  decision  to  use  the  term  “Major  Weapon  System” 
in  the  statute  has  several  advantages  over  mandating  a  PSM  requirement 
solely  for  Major  Defense  Acquisition  Programs  (MDAP)  (see  Figure  4  for 
statutory  versus  DoDI  5000.02  differences  in  definitions).  First,  it  ensures 
PSMs  will  be  assigned  to  MDAPs,  but  does  not  limit  the  positions  only  to 
those  programs,  thus  ensuring  PSMs  can  be  assigned  to  other  programs 
not  meeting  MDAP  designation  dollar  thresholds.  Second,  it  communicates 
the  intent  that  the  position  of  PSM  and  its  inherent  responsibilities  are  for 
the  life  of  the  program,  and  do  not  culminate  at  system  fielding.  Third,  once 
trained,  this  broader  pool  of  experienced  personnel  capable  of  serving  in 
a  PSM  capacity  affords  the  Services  greater  flexibility  in  hiring  the  right 
individuals  to  serve  as  PSMs  in  the  critically  important,  highly  visible  MDAPs. 


BETTER  MANAGED  WEAPON  SYSTEM  SUPPORT 

An  Aviation  Week  feature  article  (Tegtmeier,  2009)  highlighted  the 
coming  shift  in  military  sustainment  support.  Section  805  of  the  bill 
fundamentally  changes  the  way  industry  and  government  manage  major 
weapon  system  support,  according  to  Lynn  Williams,  a  staff  member  of  the 
Flouse  Armed  Services  Committee.  Section  805  of  the  bill  fundamentally 
shifts  high-level  aftermarket  responsibility  to  the  government  and  takes 
over  some  resource  allocation  from  private  industry.  It  also  requires  that 
each  major  weapon  system  be  supported  by  a  product  support  manager 
who  is  a  member  of  the  armed  services  or  a  full-time  employee  of  the 
Defense  Department.  By  considering  sustainment  requirements  in  early 
acquisition  discussions,  Williams  believes,  as  do  so  many  others,  that 
product  support  costs  should  actually  decrease  (Tegtmeier,  2009). 


ENHANCED  CREDIBILITY  OF  PBL  ASA  WEAPON  SYSTEM  PRODUCT 
SUPPORT  STRATEGY 

The  PSM  will  play  a  key  role  in  implementation  of  PBL  and  assembling 
the  team  charged  with  developing  and  executing  that  product  support 
strategy.  Both  the  Navy  (J.  Heron,  personal  communication,  December  7, 
2009)  and  the  Army,  for  example,  recognized  this  well  before  the  legislation 
was  signed,  the  latter  stating,  “a  PBL  team  should  be  formed  to  manage  the 
PBL  effort.  The  team,  led  by  the  PM  or  the  PM’s  designated  product  support 
manager  (PSM),  shall  consist  of  government  and  private-sector  functional 
experts  and  shall  include  all  appropriate  stakeholders,  including  warfighter 
representatives”  (Department  of  the  Army,  2009,  pp.  20-21). 
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GOVERNMENT  AND  INDUSTRY  ROLES  CLARIFIED 

Although  both  the  June  2009  Defense  Acquisition  Guidebook  and  the 
March  2005  Performance  Based  Logistics:  A  Program  Manager’s  Product 
Support  Guide  address  both  the  PSM  and  PSI,  as  guidebooks  they  lack  the 
authority  of  either  policy  or  statute.  Even  so,  many  misconceptions  about 
PSI  and  PSM  organizations  and  responsibilities  persist.  "What  [Congress]  is 
trying  to  do  is  clarify.. .that  we  have  an  inherently  governmental  function;  it  is 
performed  by  this  product  support  manager;  and  here  are  the  elements  of 
that  function”  (Munoz,  2009).  Section  805  reiterates  that  the  PSM  position 
is  an  inherently  governmental  role.  The  PSM  therefore  directly  supports 
the  PM  and  retains  oversight  of  PBL  implementation.  The  PSI,  on  the  other 
hand,  can  come  from  both  government  and  industry,  and  supports  both 
the  PM  and  PSM  by  integrating  (and  in  some  cases  providing)  sources  of 
product  support. 


PM-PSM  RELATIONSHIP  BETTER  UNDERSTOOD 

PMs  are  responsible  for  “ensuring  product  support  integration  as  a 
continuous  and  collaborative  set  of  activities  that  establish  and  maintain 
readiness  and  the  operational  capability  of  a  system,  subsystem,  or  end- 
item  throughout  its  life  cycle”  (Department  of  the  Air  Force,  2009b,  p.  50). 
Moreover,  "the  PM  shall  ensure  that  integrated  logistics  support  objectives 
are  considered  and  introduced  as  early  as  practical  with  a  far-reaching  life 
cycle  view  concerning  logistics  design  and  supportability  of  the  system” 
(Department  of  the  Air  Force,  2009b,  p.  113).  Although  the  PM  is  ultimately 
accountable  for  LCM  of  the  system,  the  senior  program  logistician,  among 
others  in  a  program  office,  is  responsible  for  development  of  long-term 
support  and  sustainment  planning.  The  PSM,  like  all  life  cycle  logisticians, 
is  charged  with  “translating  warfighter  performance  requirements  into 
tailored  product  support  spanning  the  system  life  cycle”  (DAU,  2009,  p. 
76).  To  achieve  this,  the  PSM  must  be  a  strong  proponent  of  LCM  principles, 
objectives,  and  implementation,  articulating  the  importance  of  long-term 
product  support  considerations  as  design  trade-offs  are  made  during 
system  development.  The  synergy,  collaboration,  and  integration  required 
between  the  PSM  and  the  PM  in  terms  of  successfully  executing  LCM  and 
product  support  responsibilities  are  unmistakable. 


CLEARLY  ARTICULATED  EXPECTATIONS 

DoD  defines  LCL  as  “the  ability  to  plan,  develop,  implement,  and 
manage  comprehensive,  affordable,  and  effective  systems  support 
strategies. ..encompass(ing)  the  entire  system’s  life  cycle,  including 
acquisition  (design,  develop,  test,  produce,  and  deploy),  sustainment 
(operations  and  support),  and  disposal”  (DoD,  2008b,  p.  16).  The  similarities 


19  6 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


to  the  expectations  outlined  by  Congress  in  the  FY2010  NDAA  and  by  DoD 
in  the  November  2009  Product  Support  Assessment  report  for  the  PSM  are 
clear,  and  as  would  reasonably  be  expected,  not  coincidental.  PSMs,  by  and 
large,  will  be  drawn  from  among  the  best,  brightest,  and  most  expert  life 
cycle  logisticians  in  the  department.  The  key  is  to  ensure  they  are  trained, 
equipped,  and  have  the  resources,  tools,  and  fortitude  to  successfully 
address  the  challenges,  responsibilities,  and  expectations  levied  upon  them. 


Recommendations 

Reformed  [product  support]  stewardship— driven  by  improving 
product  support  and  achieving  more  cost-effective  weapon  system 
readiness  outcomes— requires  a  life-cycle  management  focus, 
committed  leadership,  and  cooperative  efforts  from  the  operational, 
acquisition,  and  logistics  communities.  (DoD,  2009b,  p.  3) 

Thus  far,  this  article  has  outlined  the  intent,  importance,  and 
implications  of  the  Section  805  provisions  of  the  FY2010  NDAA  legislation 
regarding  LCM  and  product  support.  Developing  the  implementation 
policy  is  the  responsibility  of  the  Office  of  the  Secretary  of  Defense,  while 
implementation  of  the  provisions  of  that  policy  is  the  responsibility  of 
Components  and  their  materiel,  systems,  and/or  logistics  commands; 
however,  the  following  recommendations  are  provided  from  the  author’s 
perspective  as  a  career  logistician,  to  facilitate  PSM  policy  implementation 
and  offer  some  thoughts  on  things  it  will  take  in  the  areas  of  LCM  and 
product  support  to  more  effectively  support  and  sustain  the  weapon 
systems  with  which  we  defend  our  nation. 


RECOMMENDATION  NO.  1:  DoD  POLICY  CONSIDERATIONS 

Section  805  requires  the  Secretary  of  Defense,  specifically  the  Cffice 
of  the  Under  Secretary  of  Defense  (Acquisition,  Technology  &  Logistics),  to 
issue  comprehensive  LCM  guidance  and  develop  product  support  strategies 
within  180  days  after  enactment.  As  part  of  this  policy,  DoD  should  consider: 

Updating  DoDD  5000.01,  DoDI  5000.02,  the  Defense  Acquisition  Guidebook, 
and  Performance  Based  Logistics:  A  Program  Manager's  Product  Support  Guide  to 
encapsulate  PSM  roles,  responsibilities,  and  relationship  to  the  PM.  This  guidance 
should  also  include  additional  tools,  resources,  and  guidance  to  support 
the  PSM  in  performing  duties,  including  creation  of  a  new  Enclosure  13  to 
DoDI  5000.02  to  address  many  of  the  recommendations  from  the  Product 
Support  Assessment  report.  Several  to  consider  include  establishing 
Sustainment  Readiness  Levels  (SRL)  akin  to  existing  Technology  Readiness 
Levels,  mandating  standardized  Independent  Logistics  Assessments 
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(ILA)  at  regular  intervals  in  a  program  life  cycle,  and  ensuring  regular 
post-fielding  program  reviews  and  evaluation  of  sustainment  funding, 
readiness,  and  sustainment  outcome  metric  achievement. 

PSA/I  location  and  reporting  chain.  Ideally  the  PSM  would  be  assigned  to  the 
program  office  and  report  directly  to  the  program  manager.  Although  in 
some  instances  it  may  be  more  advantageous  to  have  the  PSM  collocated 
with  a  depot,  logistics  command/center,  or  field  support  activity 
(authorized  by  DoDI  5000.02)  (DoD,  2008a,  p.  72),  ensuring  strategic 
PM-PSM  alignment  and  linkage  of  long-term  product  support  objectives, 
direction,  and  strategy  development  are  imperative,  and  would  be  best 
enabled  by  a  direct  reporting  chain  to  the  PM,  regardless  of  location. 


RECOMMENDATION  NO.  2:  ESTABLISH  RIGOROUS  REQUIREMENTS  FOR 
PSM  SELECTION 

To  ensure  only  the  most  qualified  personnel  are  selected  by  the  military 
services  to  serve  as  PSMs,  update  the  December  21,  2005,  issuance  of  DoDI 
5000.66,  Operation  of  the  Defense  Acquisition,  Technology,  and  Logistics 
Workforce  Education,  Training,  and  Career  Development  Program  (DoD, 
2005,  p.  3)  to: 

•  Designate  the  PSM  position  as  both  a  Critical  Acquisition 
Position  (CAP)  and  a  Key  Leadership  Position  (KLP)  for  all 
major  weapon  systems,  including  Major  Defense  Acquisition 
Programs/Major  Automated  Information  Systems  (MDAP/ 

MAIS)  to  reflect  the  new  statutory  authority  given  in  the 
FY2010  NDAA. 

•  Add  the  PSM  as  a  position  that  should  be  considered 
for  designation  as  a  KLP  for  significant  non-MDAPs, 
recognizing  the  key  role  the  PSM  plays  in  executing  LCM 
and  program  sustainment  across  the  system  life  cycle. 

•  Identify  specific  and  rigorous  experience,  training, 
and  education  requirements  that  go  beyond  existing 
requirements  for  Level  III  LCL  certification.  In  addition  to 
other  DoDI  5000.66  CAP/KLP  requirements,  these  should 
include: 

o  At  least  8  years’  acquisition  experience,  which  includes 
at  least  6  years  in  LCL,  with  at  least  2  years  in  a  program 
office  or  similar  organization, 
o  Level  III  certification  in  LCL. 
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RECOMMENDATION  NO.  3:  EXPAND  OPPORTUNITIES  FOR  MILITARY 
PERSONNEL  TO  SERVE  AS PSM 

Undertake  initiatives  to  expand  the  number  of  uniformed  military 
personnel  in  the  LCL  career  field  to  ensure  a  pipeline  of  talented, 
experienced,  trained,  and  certified  personnel  possessing  both  acquisition 
and  operational  backgrounds  is  available  to  fill  key  PSM  positions,  along 
with  their  civilian  counterparts.  According  to  DoD  Instruction  5000.66, 
the  Under  Secretary  of  Defense  (Acquisition,  Technology  &  Logistics) 
shall  “identify  appropriate  career  paths  for  civilian  and  military  personnel 
[emphasis  added]  in  the  AT&L  Workforce  in  terms  of  education,  training, 
experience,  and  assignments  necessary  for  career  progression  to  the  most 
senior  AT&L  positions”  (DoD,  2005,  p.  4).  Moreover,  the  instruction  goes  on 
to  task  the  heads  of  DoD  Components  (acting  through  their  Component 
Acquisition  Executives)  to  not  only  “provide  opportunities  for  both  civilian 
and  military  members  [emphasis  added]  of  the  AT&L  Workforce  to  acquire 
the  education,  training,  and  experience  necessary  to  qualify  for  senior 
positions”  (DoD,  2005,  p.  4),  but  also  to  “assign  military  officers  to  provide 
a  balance  between  career-broadening  experience  and  sufficient  time  in 
each  position  to  ensure  accountability,  responsibility,  and  stability”  (DoD, 
2009b,  p.  68).  Regrettably,  only  931  military  personnel  are  assigned  to 
DoD  LCL  positions,  representing  just  6  percent  of  the  coded  positions 
in  the  career  field  (Figure  5).  Of  these,  none  come  from  the  Army  or  the 
Fourth  Estate  (Defense  Agencies),  and  a  total  of  just  64  military  personnel 
are  currently  certified  at  Level  III.  Ensuring  a  sufficiently  robust  pool  of 
experienced,  talented,  trained,  and  operationally  experienced  uniformed 
military  personnel  to  complement  their  civilian  counterparts  will  go  a  long 
way  toward  ensuring  the  success  of  the  PSM  requirement. 


RECOMMENDATION  NO.  4:  IMPLEMENT  DoD  PRODUCT  SUPPORT 
ASSESSMENT  HUMAN  CAPITAL  RECOMMENDATIONS 


DoD  should  aggressively  implement  key  recommendations  contained 
in  the  November  2009  DoD  Product  Support  Assessment  report  in  order 
to  provide  the  PSM,  the  PM,  the  DoD  Components,  and  the  department 
the  ability  to  successfully  achieve  congressionally  mandated  Section 
805  requirements,  and  in  the  process,  increase  competition,  enhance 
performance  based  life-cycle  product  support,  reduce  life-cycle  costs,  and 
improve  weapon  system  performance  outcomes.  Development  of  more 
robust  analytical  tools,  policies,  and  processes  for  performing  business 
case  analyses,  better  oversight  of  operations  and  sustainment  costs,  and 
expanding  public-private  partnering  are  all  strategic  initiatives,  among 
many  others,  which  complement  Section  805  direction.  This  would  also 
include  aligning  human  capital  report  recommendations  with  PSM  roles, 
responsibilities,  and  authority  to  “identify  new  or  modified  product  support 
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FIGURE  5.  AT&L  LIFE  CYCLE  LOGISTICS  FUNCTIONAL  AREA  AS 
OF  DECEMBER  31,  2009 


Career  Level 
Achieved 

Workforce 

Category 

Army 

Navy 

Air 

Force 

4th 

Estate 

Total 

Level  I 

Civilian 

1,901 

745 

348 

12 

3,006 

Military 

- 

114 

104 

- 

218 

Subtotal 

1,901 

859 

452 

12 

3,224 

Level  II 

Civilian 

1,560 

1,169 

466 

29 

3,224 

Military 

- 

57 

40 

- 

97 

Subtotal 

1,560 

1,226 

506 

29 

3,321 

Level  III 

Civilian 

2,214 

1,553 

261 

61 

4,089 

Military 

- 

56 

8 

- 

64 

Subtotal 

2,214 

1,609 

269 

61 

4,153 

No  Level  Achieved/ 

Civilian 

2,637 

966 

465 

20 

4,088 

Unknown 

Military 

- 

237 

315 

- 

552 

Subtotal 

2,637 

1,203 

780 

20 

4,640 

Totals 

8,312 

4,897 

2,007 

122 

15,338 

{Source:  AT&L  Workforce  Data  Mart) 

competencies  and  proficiencies  driven  by  proposed  (product  support 
assessment)  strategy,  policy,  and  process  changes”  and  to  “incorporate 
new  or  modified  product  support  competencies  into  DoD  and  industry 
logistics,  and  acquisition  workforce  career  field  training,  recruitment,  and 
retention  strategies”  (DoD,  2009b,  p.  69),  including: 

•  Focusing  on  enhancing  professional  development,  including 
greater  emphasis  on  the  seven  key  life  cycle  logistics 
competencies  outlined  in  the  May  2008  DoD  Logistics 
Human  Capital  Strategy  (Logistics  Design  Influence, 

Integrated  Logistics  Support  Planning,  Product  Support 

&  Sustainment,  Configuration  Management,  Reliability 
&  Maintainability  Analysis,  Technical/Product  Data 
Management,  and  Supportability  Analysis)  (DoD,  2008b, 
p.  4)  and  underlying  proficiencies,  which  serve  as  the 
building  blocks  for  each  competency. 

•  Developing  highly  capable,  highly  effective  PSMs  with  a 
truly  strategic,  enterprise-level  perspective.  Such  individuals 
would  possess  the  experience  of  what  the  DoD  Logistics 
Human  Capital  Strategy  terms  a  “multi-  faceted  logistician 
with  expertise  in  many  segments  and  knowledge  of  the 
logistics  process  end-to-end;  knowledge  of  business  or 
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other  fields;  executive  training;  and  multi-component 
experience”  (DoD,  2008b,  p.  4). 

Establishing  more  rigorous,  competency-based  defense 
acquisition  workforce  certification  training,  including  new 
DAD  courses  focusing  on  RAM,  supportability  analysis, 
technical  data/product  data  management,  and  expanded 
performance  based  life-cycle  product  support  and 
sustainment  courseware  to  strengthen  preparation  of  future 
PSMs  during  their  acquisition  professional  development. 
Identifying  executive-level  PSM  competencies  and 
development  of  400-level  training  for  PSMs  comparable  to 
existing  PMT  401  and  PMT  402  training  currently  available 
for  senior  program  managers.  Moreover,  because  product 
support  is  broader  than  LCL,  this  would  entail  going  beyond 
a  logistics  audience  and  identifying  executive-level  product 
support  competencies  and  training  for  other  acquisition 
functional  career  fields. 


RECOMMENDATION  NO.  5:  ALIGN  EXISTING  POLICY  WITH  NEW 
STATUTORY  REQUIREMENTS 

First  and  foremost,  DoD  Section  805  implementation  guidance 
will  need  to  address  a  variety  of  LCM  enablers  related  to  competition, 
best  value  determination,  resource  allocation,  business  case  analysis, 
strengthening  outcome-based  product  support  implementation,  long¬ 
term  sustainment  oversight,  and  of  course,  PSM  roles,  responsibilities,  and 
authority.  In  addition,  with  codification  of  the  PSM  position  into  law.  Service- 
specific  guidance  such  as  direction  contained  in  Air  Force  Instruction 
61-101,  “identify  a  product  support  integrator  as  a  single  point  of  contact 
prior  to  program  initiation”  and  “the  product  support  integrator  will  be 
military  or  government  civilian  personnel  unless  otherwise  approved  and 
documented  as  part  of  program  planning”  (Department  of  the  Air  Force, 
2009b,  p.  113)  should  be  revised  to  reflect  the  fact  that  the  PSM  must 
be  either  military  or  a  government  civilian,  and  at  the  same  time,  PSI 
responsibilities  under  a  PBL  arrangement  can  be  performed  by  either  a 
governmental  or  industry  organization.  Section  805  of  the  FY2010  NDAA 
clearly  states  “the  term  ‘product  support  integrator’  means  an  entity  within 
the  Federal  Government  or  outside  the  Federal  Government  [emphasis 
added]  charged  with  integrating. ..sources  of  product  support,  both  private 
and  public,  defined  within  the  scope  of  a  product  support  arrangement” 
(NDAA,  2009a). 
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Conclusions 

Planning  for  Operations  and  Support  and  the  estimation  of  total 
ownership  costs  shall  begin  as  early  as  possible.  Supportability,  a 
key  component  of  performance,  shall  be  considered  throughout 
the  system  life  cycle.  (DoD,  2007,  p.  10) 

DoD  is  at  a  critical  juncture.  Supporting  and  sustaining  increasingly 
complex,  often  aging  weapon  systems  in  an  era  of  budgetary  austerity, 
and  faced  with  a  variety  of  threats  and  challenges  from  both  state  and 
non-state  actors,  the  department  must  leverage  LCM  processes,  practices, 
and  policies,  coupled  with  performance  based  life-cycle  product  support 
sustainment  strategies  to  preclude  degraded  readiness  and  upward 
spiraling  support  costs.  By  including  Section  805  in  the  FY2010  NDAA, 
Congress  has  made  it  clear  where  it  stands  on  these  issues,  and  who  is 
responsible  for  addressing  them.  “The  true  decision-making  authority  lies 
with  the  product  support  managers,  who  determine  'allocation  decisions, 
strategy  decisions,  doing  the  business  case  analysis  to  determine  the  best 
approach  for  sustaining  the  weapons  system’”  (Munoz,  2009). 

The  confluence  in  recent  months  of  formal  statutory  recognition  of 
the  PSM,  issuance  of  a  DoD  Logistics  Human  Capital  Strategy  capturing 
the  vision  and  required  competencies,  recognition  by  the  department 
that  performance  based  logistics  strategies  must  be  strengthened  and 
broadened  to  more  effectively  inculcate  product  support  (of  which  logistics 
is  an  important,  but  by  no  means  exclusive  subset),  and  issuance  of  a  year¬ 
long  DoD  Product  Support  Assessment  report  all  point  to  the  same  desired 
outcome:  genuine  LCM,  which  delivers  sustained  long-term  weapon  system 
readiness  while  optimizing  life-cycle  costs.  The  stars  are  aligned  for  product 
support  success  like  never  before. 
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HOW  WELL  ARE  PMs  DOING? 

INDUSTRY  VIEW 
OF  DEFENSE 
PROGRAM  MANAGER 
COUNTERPARTS 


f  Roy  L.  Wood 

Large,  complex  defense  acquisition  programs  have  been 
plagued  by  cost  overruns,  delayed  schedules,  and  subpar 
performance.  Much  of  the  responsibility  has  been  attributed  to 
weaknesses  in  competencies  of  government  program  managers 
(PM).  This  study  provides  a  new  perspective  on  government  PM 
competencies  by  surveying  defense  industry  managers  who 
work  with  the  government  PMs.  Data  gathered  from  a  survey 
of  146  industry  managers  rated  the  importance  of  common 
PM  competencies  and  assessed  how  well,  from  their  perspec¬ 
tive,  their  government  counterparts  met  those  competencies. 
The  data  also  revealed  several  insights,  including  a  conclusion 
that  government  PM  performance  on  several  key  technical 
skills  may  need  improvement.  The  results  of  this  study  will  be 
useful  In  assessing  training  and  development  strategies  for 
government  PMs. 


Keywords:  Program  Management,  Competencies, 
Skills,  Leadership,  Organizational  Behavior 
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The  Government  Accountability  Office  (GAO),  an  independent 
investigative  arm  of  the  U.S.  Congress,  reported  in  2008  that  the 
Department  of  Defense  (DoD)  had  $1.6  trillion  in  commitments  for  weapons 
systems  acquisition  programs,  with  estimated  cost  growth  of  $295  billion 
and  average  schedule  delays  of  21  months  (GAO,  2008,  p.  4).  Indeed, 
for  the  past  several  decades,  news  reports  of  $600  toilet  seats,  poor 
performance  of  battlefield  equipment,  and  cancelled  programs  have  been 
all  too  commonplace  (Besselman,  Arora,  &  Larkey,  2000;  Samuel,  2003). 
The  Defense  Acquisition  Program  Assessment  (DAPA)  Report  of  2006 
(Kadish,  2006)  more  recently  asserted  that: 

Both  Congress  and  the  Department  of  Defense  senior  leadership 
have  lost  confidence  in  the  capability  of  the  Acquisition  System  to 
determine  what  needs  to  be  procured  or  to  predict  with  any  degree 
of  accuracy  what  things  will  cost,  when  they  will  be  delivered,  or 
how  they  will  perform,  (p.  1) 

DoD  program  managers  (PM)  have  come  to  bear  much  of  the 
responsibility  for  these  overruns  in  cost  and  schedule  (GAO,  2005;  Kadish, 
2006).  In  the  DAPA  report,  "program  manager’s  expertise”  was  identified 
as  one  of  the  top  five  issues  contributing  to  the  poor  program  performance 
(Kadish,  2006,  p.  3);  and  in  2008,  the  GAO  commented  that  the  DoD 
needed  to  “strengthen  training  and  career  paths  as  needed  to  ensure 
program  managers  have  the  right  qualifications  for  running  the  programs 
they  are  assigned  to”  (Sullivan,  2008,  p.  16).  Part  of  the  solution  to  improving 
acquisition  program  outcomes,  then,  may  lie  in  identifying  and  improving 
specific  competencies  of  the  program  managers  themselves. 

This  is  not  a  simple  undertaking.  Most  who  understand  the  job  of 
the  program  manager  in  defense  acquisitions  appreciate  the  breadth 
of  knowledge,  skills,  and  abilities  he  or  she  must  possess  in  several 
competencies,  including: 

•  The  PM  must  be  technically  competent,  able  to  manage 
technology  and  system  engineering  as  well  as  software  and 
information  systems,  and  understand  manufacturing  and 
industrial  processes. 

•  The  PM  must  demonstrate  key  business  competencies  such 
as  financial  management,  contracting,  and  cost  estimating. 

•  The  PM  must  exercise  leadership  and  management 
competencies  in  developing  and  executing  the  program 
strategy,  managing  core  processes,  and  dealing  with  the 
day-to-day  management  challenges  of  a  large,  complex 
program. 
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Fox  and  Miller  (2006)  summed  up  the  need  for  this  broad  and 
comprehensive  PM  competency  set  by  stating: 

Managing  [a  large  complex  project]  is  more  than  a  science;  it 
is  a  continually  evolving  art...  Managers  must  augment  a  strong 
foundation  of  conventional  management  skills  in  planning, 
organizing,  and  controlling,  with  knowledge  of  the  requirements, 
resources,  and  constraints  of  a  specific  project  as  it  progresses, 
(p.  109) 

Given  the  expansive  portfolio  of  required  competencies,  one  might 
question  whether  some  competencies  are  more  important  to  program 
success  than  others.  For  example,  research  by  Bauer  (2006),  asserted  that 
management  competence  is  more  important  for  defense  and  aerospace 
industry  program  managers  than  fec/in/ca/ competence.  Likewise,  Gadeken 
(2004)  reported  research  from  government  PM  self-assessments  that 
suggested  leadership  and  management  skills  were  more  important  than 
technical  skills. 

Once  critical  PM  competencies  are  identified,  the  next  logical  inquiry 
would  be  into  which  of  those  competencies  PMs  might  be  demonstrating 
systemic  weaknesses.  If  important  competencies  can  be  identified, 
and  weaknesses  among  those  discovered,  perhaps  focused  training 
and  development  remediation  can  be  applied  to  improve  these  PM 
competencies,  and  thus  impact  program  performance. 

To  try  to  address  this  opportunity,  this  exploratory,  quantitative 
study  began  with  a  set  of  35  specific  technical/business  (“hard  skill”) 
and  leadership/management  (“soft  skill”)  competencies  and  attempted 
to  determine  which  were  perceived  as  most  important  in  contributing 
to  program  success,  and  how  well  PMs  were  performing  against  those 
competencies.  To  minimize  bias  associated  with  self-surveys,  a  sampling 
of  experienced  defense  industry  managers  was  invited  to  participate  in  a 
survey  that  asked  them  to  objectively  assess  the  skills  and  abilities  of  their 
government  counterparts.  This  approach  provided  a  unique  perspective 
on  government  PM  competencies  that  had  not  been  explored  previously 
in  the  literature. 


Research  Method 

While  most  competency  studies  in  the  literature  involved  collecting 
data  from  PM  self-surveys  or,  in  some  cases,  surveys  of  PM  supervisors 
(Besner  &  Flobbs,  2006;  Cheng,  Dainty,  &  Moore,  2005;  Gehring,  2007; 
Muzio,  Fisher,  Thomas,  &  Peters,  2007),  this  study  attempted  to  use  a 
fresh  approach.  Here,  defense  Industry  PMs  were  surveyed  and  asked  to 
provide  their  perceptions  and  assessments  of  the  core  competencies  of 
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their  government  counterparts.  Industry  PMs  were  selected  because  they 
are  in  a  singularly  distinctive  position  to  be  able  to  assess  their  government 
PM  equivalents.  Government  and  industry  PMs  typically  work  very  closely 
together  on  defense  programs,  providing  the  unique  opportunity  for  these 
industry  managers  to  closely  observe  their  government  PM  counterparts 
and  contribute  rare  and  valuable  insights  to  this  study. 

Key  competencies  of  the  government  PMs  were  measured  using  a 
survey  instrument  originally  developed  by  Golob  (2002).  The  survey 
instrument  was  based  in  part  on  Project  Management  Institute  (PMI) 
competencies  and  modified  for  the  purposes  of  this  investigation.  Survey 
validity  and  reliability  were  verified  through  expert  evaluation,  pilot  surveys, 
and  standard  statistical  methods. 

The  competencies  included  20  technical/business,  or  “hard  skills,” 
and  15  leadership/management,  or  "soft  skills,”  as  shown  in  Table  1. 
Survey  participants  were  asked  to  address  two  questions.  First,  which 
government  project  management  competencies  among  the  35  given  are 
most  important  for  program  success?  Participants  responded  to  the  list  of 
competencies,  rating  the  relative  contribution  of  each  to  program  success. 
Each  competency  was  listed  on  the  questionnaire  with  Likert  scale  choices 
of  Very  Important,  Important,  Neutral,  Unimportant,  or  Very  Unimportant. 
The  second  research  question  put  to  the  industry  managers  was  how  well 
government  PM  counterparts  performed  against  each  competency.  The 
Likert  scale  observations  included  ratings  of  Expert,  Good,  Average,  Fair, 
Poor,  and  a  no-response  choice. 

Participating  in  the  survey  were  146  industry  managers,  providing  a 
good  statistical  basis  for  insights  into  PM  competencies.  Demographic 
information  from  the  survey  revealed  that  the  sample  included  a  large 
proportion  of  senior  industry  managers  with  substantial  experience 
managing  complex  defense  programs.  The  survey  demographics  are 
depicted  in  the  Figure.  The  data  also  show  that  the  industry  managers  had 
frequent  contact  with  their  government  counterparts,  lending  credence  to 
their  observations. 


Results 


COMPETENCY  IMPORTANCE  TO  PROGRAM  SUCCESS 

To  address  the  importance  of  each  competency  to  program  success, 
the  mean  scores  for  each  competency  were  compared  and  rank  ordered.  A 
higher  average  score  indicated  that  the  industry  managers  perceived  this 
particular  competency  to  be  a  more  important  determinant  of  program 
success.  Table  1  shows  the  means  ranking  of  the  importance  data.  Since 
the  survey  was  based  on  a  mature  set  of  widely  accepted  competencies. 
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TABLE  SURVEY  COMPETENCIES  AND  DEFINITIONS 


Hard  Skills  (C1-C20) 

1. 

Determine  program  goals,  requirements,  and  specifications 

2. 

Determine  program  scope  and  deliverables 

3. 

Technical  ability 

4. 

Document  program  constraints  that  could  affect  program  completion 

5. 

Document  program  assumptions 

6. 

Define  program  strategy  or  alternative  approaches 

7. 

Quality  assurance 

8. 

Identify  resources  requirements 

9. 

Develop  a  budget 

10. 

Create  a  work  breakdown  structure  (WBS) 

11. 

Develop  a  schedule 

12. 

Develop  a  resource  management  plan 

13. 

Establish  program  controls  comparing  actual  against  planned  performance 

14. 

Develop  program  plan 

15. 

Communicate  program  status 

16. 

Measure  program  performance  to  identify  program  trends  and  variances 

17. 

Implement  corrective  action 

18. 

Implement  change  control 

19. 

Respond  to  risk 

20. 

Conduct  administrative  closure  of  the  program  upon  completion 

Management/Leadership  (Soft  Skill)  Competencies  (CS1-15) 

1. 

Project  leadership 

2. 

Flexibility  to  adapt  and  deal  with  situations  and  manage  expectations 

3. 

Sound  business  judgment 

4. 

Trustworthiness 

5. 

Communication  style  presents  clear  and  unambiguous  information  without  bias 

6. 

Listening  skills 

7. 

Setting  and  managing  expectations 

8. 

Negotiations 

9. 

Issue  and  conflict  resolution 

10. 

Organizational  skills 

11. 

Coaching 

12. 

Facilitation 

13. 

Decision  making 

14. 

Problem  solving 

15. 

Team  building 
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FIGURE  1.  SURVEY  DEMOGRAPHICS 


Program  Phase 


Communication  with  Government  Counterpart 


industry  managers  rated  most  of  the  competencies  very  highly  in 
importance  to  project  success. 

The  highest  rated  competencies  represented  a  relatively  even  mix  of 
technical  and  soft  skills.  The  most  valued  hard  skills  were  the  ability  to 
determine  program  goals  and  deliverables  and  develop  a  program  budget. 
These  results  were  not  surprising.  Among  others,  Pinkerton  (2003,  p.  53) 
pointed  out  that  the  first  criterion  for  project  success  is  to  have  clearly 
defined  goals  and  objectives.  It  is  important  for  the  government  to  specify 
the  deliverables  from  the  project,  and  it  is  equally  important  for  industry, 
because  deliverables  define  the  government’s  expectations  in  concrete 
terms.  Similarly,  a  sound  program  budget  is  important  to  match  resources 
to  goals  and  deliverables. 

The  most  highly  rated  soft  skills  included  trustworthiness,  project 
leadership,  and  decision  making.  Trust  and  trustworthiness  are  keys  to 
proper  organizational  and  interorganizational  functioning  and  have  been 
documented  in  the  literature  (Jehn  &  Mannix,  2001;  Joseph  &  Winston, 
2005;  Wells  &  Kipnis,  2001).  Trust  may  be  particularly  important  in  large, 
complex  projects  where  not  every  expectation  can  be  instantiated  in  the 
government-industry  contract.  Trust  and  understanding  between  the 
government  and  industry  managers  are  essential  to  minimize  conflict,  foster 
cooperation,  and  jointly  succeed. 

Similarly,  project  leadership  in  a  complex  defense  project  is  required 
to  establish  the  vision  and  goals,  motivate  the  team,  and  gain  commitment 
to  program  success.  The  third-ranking  attribute,  PM  decision  making,  is 
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TABLE  2.  COMPETENCY  SCORES  FOR  IMPORTANCE 


Competency 

Designation 

Competency  Description 

M 

SD 

Ranking 
by  Mean 

C  1 

Determine  program  goals 

4.86 

0.345 

1 

C  2 

Determine  program  deliverables 

4.75 

0.478 

2 

C  3 

Technical  ability 

4.14 

0.533 

28 

C  4 

Document  constraints 

4.47 

0.634 

9 

C  5 

Document  assumptions 

4.18 

0.599 

26 

C  6 

Define  program  strategy 

4.38 

0.624 

15 

C  7 

Quality  assurance 

4.10 

0.782 

29 

C  8 

Identify  resources  needs 

4.30 

0.626 

22 

C  9 

Develop  a  budget 

4.62 

0.578 

5 

C  10 

Create  a  WBS 

3.85 

0.861 

34 

C  11 

Develop  a  schedule 

4.53 

0.645 

8 

C  12 

Develop  a  resource  mgt  plan 

4.02 

0.815 

31 

C  13 

Establish  program  controls 

4.44 

0.664 

11 

C  14 

Develop  program  plan 

4.37 

0.752 

18 

C  15 

Communicate  status 

4.27 

0.638 

23 

C  16 

Measure  performance 

4.35 

0.594 

20 

C  17 

Implement  corrective  action 

4.47 

0.553 

10 

C  18 

Implement  change  control 

4.31 

0.739 

21 

C  19 

Respond  to  risk 

4.41 

0.607 

13 

C  20 

Administrative  closure 

3.66 

0.771 

35 

C  SI 

Project  leadership 

4.65 

0.493 

4 

C  S2 

Flexibility 

4.42 

0.549 

12 

C  S3 

Business  judgment 

4.36 

0.560 

19 

C  S4 

Trustworthiness 

4.75 

0.452 

3 

C  S5 

Communication  style 

4.21 

0.528 

25 

C  S6 

Listening  skills 

4.27 

0.567 

24 

C  S7 

Set  and  manage  expectations 

4.40 

0.557 

14 

C  S8 

Negotiation 

4.38 

0.623 

17 

C  S9 

Issue  and  conflict  resolution 

4.16 

0.547 

27 

C  SIO 

Organizational  skills 

4.05 

0.608 

30 

C  Sll 

Coaching 

4.01 

0.712 

32 

C  S12 

Facilitation 

3.85 

0.709 

33 

C  S13 

Decision  making 

4.60 

0.533 

6 

C  S14 

Problem  solving 

4.38 

0.590 

16 

C  S15 

Team  building 

4.54 

0.578 

7 

Note.  C  =  Technical  Skill;  CS  =  Soft  Skill 
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also  important  since  most  decisions  require  the  PM  to  delicately  balance 
program  goals  and  powerful  stakeholder  interests.  Complex  program 
decisions  reflect  organizational  behavior  factors  involving  resolution  of 
conflicting  program  goals,  and  avoidance  of  uncertainties  that  can  create 
program  risk  (Cyert  &  March,  1958). 


Competency  Performance 

A  similar  means  analysis  was  also  conducted  to  allow  rank  ordering 
of  the  data  for  the  second  question  of  whether  the  government  PMs  were 
perceived  as  meeting  the  expectations  for  each  of  the  competencies  in 
the  study.  Table  2  shows  the  means  ranking  of  the  performance  data. 
Noteworthy  (and  perhaps  a  bit  disturbing)  is  that  the  performance  scores 
for  all  competencies  generally  rated  only  near  average.  This  overall  result 
can  only  be  considered  surprising  and  not  a  little  disappointing,  given  the 
high  stakes  and  inherent  expectations  that  defense  PMs  are  capable  of 
managing  billions  of  taxpayer  dollars  and  providing  critical  defense  systems 
to  the  battlefield.  The  results  seem  to  indicate  that  government  skills  could 
generally  use  additional  developmental  improvement  across  the  entire 
spectrum  of  hard  and  soft  skills. 

Closer  examination  of  the  data  nearer  the  bottom  of  the  performance 
range  reveals  items  with  lower  perceived  performance,  such  as  the  PMs’ 
ability  to  implement  change  control,  develop  a  resource  management 
plan,  or  provide  coaching.  In  absolute  terms,  these  are  important  insights 
to  areas  where  training  and  development  could  potentially  help  improve 
skills.  However,  since  many  of  the  items  have  poor  survey  assessments, 
it  becomes  important  to  weight  the  findings  by  importance  to  be  more 
discerning  of  the  areas  where  improvements  might  yield  greater  value. 


Determining  the  Competency  Gap 

In  order  to  judge  the  relative  size  of  the  performance  gap  in  PMs’ 
meeting  important  competencies,  the  results  of  the  two  rankings— 
importance  and  performance— were  compared  and  more  closely  analyzed. 
The  assessment  approach  for  this  analysis  was  based  on  the  Borich 
weighting  model  (1980).  In  this  model  (Table  3),  the  difference  between 
the  mean  assessed  ability  to  meet  a  competency  was  compared  to  the 
mean  perceived  importance  to  measure/identify  the  magnitude  of  the 
discrepancy.  This  discrepancy  score  was  then  multiplied  by  the  competency 
importance  to  garner  a  weighted  score. 

By  using  this  method,  items  with  the  largest  gap  between  importance 
and  performance  migrated  to  the  top  of  the  list,  reflecting  a  more  finely 
prioritized  list  of  important  competencies  with  larger  shortfalls.  For 


How  Well  Are  PMs  Doing?  Industry  View  of 
Defense  Program  Manager  Counterparts 


April  2010 


215 


TABLE  3.  COMPETENCY  PERFORMANCE  DATA 


Competency 

Designation 

Competency  Description 

M 

SD 

Ranking 
by  Mean 

C  1 

Determine  program  goals 

3.42 

0.911 

4 

C  2 

Determine  program  deliverables 

3.27 

1.015 

10 

C  3 

Technical  ability 

3.45 

1.043 

2 

C  4 

Document  constraints 

2.98 

1.029 

27 

C  5 

Document  assumptions 

2.97 

0.958 

29 

C  6 

Define  program  strategy 

3.03 

1.032 

24 

C  7 

Quality  assurance 

3.32 

0.816 

7 

C  8 

Identify  resources  needs 

3.04 

1.068 

23 

C  9 

Develop  a  budget 

2.90 

1.121 

31 

C  10 

Create  a  WBS 

3.05 

0.991 

21 

C  11 

Develop  a  schedule 

3.09 

1.018 

18 

C  12 

Develop  a  resource  mgt  plan 

2.86 

0.855 

34 

C  13 

Establish  program  controls 

3.00 

1.057 

25 

C  14 

Develop  program  plan 

3.13 

0.987 

14 

C  15 

Communicate  status 

3.43 

1.050 

3 

C  16 

Measure  performance 

3.35 

0.978 

5 

C  17 

Implement  corrective  action 

3.05 

1.042 

22 

C  18 

Implement  change  control 

2.68 

1.135 

35 

C19 

Respond  to  risk 

3.12 

0.943 

16 

C  20 

Administrative  closure 

2.88 

1.063 

32 

C  SI 

Project  leadership 

3.30 

1.046 

8 

C  S2 

Flexibility 

3.07 

1.075 

19 

C  S3 

Business  judgment 

2.99 

1.078 

26 

C  S4 

Trustworthiness 

3.62 

1.160 

1 

C  S5 

Communication  style 

3.22 

1.125 

11 

C  S6 

Listening  skills 

3.21 

1.029 

12 

C  S7 

Set  and  manage  expectations 

3.07 

0.976 

20 

C  S8 

Negotiation 

2.93 

1.154 

30 

C  S9 

Issue  and  conflict  resolution 

3.10 

1.025 

17 

C  SIO 

Organizational  skills 

3.21 

0.798 

13 

C  Sll 

Coaching 

2.87 

1.046 

33 

C  S12 

Facilitation 

2.98 

0.984 

28 

C  S13 

Decision  making 

3.34 

0.987 

6 

C  S14 

Problem  solving 

3.28 

0.998 

9 

C  S15 

Team  building 

3.13 

1.039 

15 

Note.  C  =  Technical  Skill;  CS  =  Soft  Skill 
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example,  using  this  method,  even  though  trustworthiness  was  rated  of 
high  importance,  it  was  deemphasized  in  the  gap  analysis  because  it  scored 
relatively  well  in  the  performance  assessment.  Conversely,  the  chosen 
method  elevated  negotiation  skiiis  to  a  higher  gap  position  even  though 
it  was  rated  in  the  middle  range  of  importance,  since  it  was  assessed  near 
the  bottom  of  PM  performance. 

Table  4  shows  the  top  10  competency  gaps  based  on  the  Borich 
analysis.  In  this  list  a  surprising  number  of  technical  skills  topped  the  list, 
including  deveiop  a  budget,  impiement  change  controi,  document  program 
constraints,  and  determine  program  deiiverabies  (Borich,  1980).  Of  the  top 
10  items,  only  two  identified  shortfalls  were  soft  skills— negof/af/on  and 
team  buiiding.  These  results  seem  contrary  to  assertions  by  Bauer  (2006) 
and  Golob  (2002)  that  soft  skills  may  be  the  most  important  to  program 


TABLE  4.  COMPETENCY  SHORTFALLS  USING  BORICH  MODEL 
(ABRIDGED  TO  TOP  10) 


Competency 

Importance 

Performance 

Difference 

Designation 

Competency 

(1) 

(P) 

1  X  (l-P) 

C  9 

Develop  a 
budget 

4.616 

2.902 

7.913 

C  2 

Determine 

program 

deliverables 

4.753 

3.268 

7.060 

C  18 

Implement 
change  control 

4.308 

2.676 

7.030 

C  1 

Determine 

4.863 

3.420 

7.016 

program  goals 

C  4 

Document 

program 

constraints 

4.466 

2.978 

6.643 

C  11 

Develop  a 
schedule 

4.527 

3.088 

6.519 

C  13 

Establish 

program 

controls 

4.438 

3.000 

6.384 

C  S15 

Team  building 

4.538 

3.132 

6.378 

C  S8 

Negotiations 

4.377 

2.927 

6.345 

C  17 

Implement 

corrective 

action 

4.466 

3.051 

6.316 
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success,  and  the  study  by  Gadeken  (2004),  which  suggested  that  defense 
PMs  should  seek  soft-skill  training. 

Conclusions 

The  current  study  appears  to  be  the  first  in  the  literature  to  explore 
the  competencies  of  Department  of  Defense  program  managers  from  the 
perspective  of  their  industry  counterparts.  The  data  allowed  for  the  ranking 
of  competencies  believed  to  contribute  most  to  program  success,  as  well  as 
assess  how  well  defense  PMs  met  those  competencies.  From  these  results,  a 
priority-ordered  list  was  developed  of  competencies  that  are  candidates  for 
improvement  through  training  and  development.  The  competencies  ranking 
in  the  top  10  for  importance  represented  a  relatively  even  mix  of  technical 
and  soft  skills,  as  did  the  raw  rankings  of  PM  performance.  However, 
when  analysis  was  done  to  discover  the  variance  between  competency 
importance  and  performance,  the  results  ranked  many  of  the  technical 
skills  at  the  top  of  the  list  of  candidates  for  improvement.  These  findings 
seem  to  refute  the  conventional  wisdom  and  may  provide  new  insights  and 
contributions  to  the  literature. 
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IMPROVING 

DEFENSE  ACQUISITION 
DECISION  MAKING 

f  COL  William  R.  Fast,  USA  (Ret.) 

This  research  investigates  evidence  and  tests  the  hypothesis 
that  the  linkages  between  the  defense  acquisition  management 
system,  the  requirements  process,  and  the  budgeting  system 
are  not  sufficiently  defined  to  enable  the  success  of  acquisition 
programs.  These  disconnects  contribute  to  weapons  systems 
cost  overruns,  schedule  delays,  and  performance  problems,  and 
are  exacerbated  by  the  ever-changing  global  security  environ¬ 
ment  and  rapid  pace  of  technological  advancement.  Through 
historical  research,  qualitative  and  quantitative  analyses,  and  a 
comprehensive  review  of  current  policies  and  procedures,  this 
research  illuminates  these  areas  of  disconnect  and  proposes 
specific  recommendations  to  fix  them. 


Keywords:  Acquisition,  Budgeting,  Decision  Making, 
Programming,  Requirements 
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The  primary  purpose  of  this  research  was  to  investigate  how  well  the 
Defense  Acquisition  Management  System  interfaces  with  the  requirements 
and  budgeting  systems  of  the  Department  of  Defense  (DoD).  The  United 
States  of  America  possesses  the  finest  weapons  systems  in  the  world. 
However,  the  same  cannot  be  said  for  the  systems  that  enable  the  Pentagon 
to  acquire  those  weapons  systems.  Cost  overruns,  schedule  delays,  and 
operational  test  failures  testify  to  numerous  severed  connections  among  the 
acquisition  management,  requirements,  and  budgeting  systems  (commonly 
referred  to  as  the  three  decision  support  systems).  The  ever-changing 
global  security  environment  and  the  rapid  pace  of  technological  change 
only  serve  to  exacerbate  these  problems. 

For  the  Pentagon  to  earn  a  reputation  for  excellence  in  acquiring 
weapons  systems,  these  decision  support  systems  must  operate  with 
far  better  coordination  and  demonstrate  that  they  can  procure  the  right 
equipment,  within  reasonable  timeframes,  and  at  affordable  prices.  This 
research  began  with  an  investigation  into  the  intricacies  of  the  acquisition 
management,  requirements,  and  budgeting  systems.  Next,  interactions 
between  these  three  decision  support  systems  were  illuminated  to  uncover 
areas  of  misalignment  and  disconnect.  Recent  initiatives  to  correct  these 
problems  were  also  identified.  Finally,  solutions  to  resolve  these  disconnects 
were  enumerated. 

Background 

A  January  2006  report  of  the  Defense  Acquisition  Performance 
Assessment  (DAPA)  described  the  three  decision  support  systems  as: 

...a  highly  complex  mechanism  that  is  fragmented  in  its  operation. 
Further,  the  findings  we  developed  indicated  that  differences 
in  the  theory  and  practice  of  acquisition,  divergent  values 
among  the  acquisition  community,  and  changes  in  the  security 
environment  have  driven  the  requirements,  acquisition,  and  budget 
processes  further  apart,  and  have  inserted  significant  instability 
into  the  acquisition  system.  In  theory,  new  weapons  systems 
are  delivered  as  the  result  of  the  integrated  actions  of  the  three 
interdependent  processes  whose  operations  are  held  together 
by  the  significant  efforts  of  the  organizations,  workforce,  and  the 
industrial  partnerships  that  manage  them.  In  practice,  however, 
these  processes  and  practitioners  often  operate  independent  of 
one  another.  Uncoordinated  changes  in  each  of  the  processes  often 
cause  unintended  negative  consequences  that  magnify  the  effects 
of  disruptions  in  any  one  area.'  (DAPA  Panel,  2006,  pp.  4-5) 
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FIGURE  1.  DoD  DECISION  SUPPORT  SYSTEMS 


Figure  1  highlights  the  areas  of  interaction  between  the  Defense 
Acquisition  Management  System,  the  Joint  Capabilities  Integration  and 
Development  System  (JCIDS),  and  the  Planning,  Programming,  Budgeting, 
and  Execution  (PPBE)  system.  Coordinated  management  decisions  at 
these  interfaces  are  essential  for  the  success  of  any  acquisition  program. 
Thus,  this  research  began  by  seeking  to  understand  the  reasons  why  these 
three  decision  support  systems  were  first  established  and  how  acquisition 
programs  are  affected  by  the  decisions  made  within  and  between  these 
systems  today. 


DEFENSE  ACQUISITION  MANAGEMENT  SYSTEM  (DAMS):  STRATIFIED 
DECISIONMAKING 

Decision  making  in  today’s  Defense  Acquisition  Management  System 
(DAMS)  can  be  traced  to  1986.  The  late  David  Packard,  then  president  of 
Hewlett-Packard,  was  selected  by  Ronald  Reagan  to  lead  the  President’s 
Blue  Ribbon  Commission  on  Defense  Management.  Better  known  as  the 
Packard  Commission,  its  interim  report  of  April  1986  recommended  the 
appointment  of  both  DoD-level  and  Service  Acquisition  Executives  (SAEs). 
The  SAEs  would  appoint  Program  Executive  Officers  (PEOs)  under  their 
authority  that  would  be  responsible  for  a  manageable  number  of  acquisition 
programs  and  project  managers.  By  design,  the  chain  of  authority  from  the 
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FIGURE  2.  THE  DEFENSE  ACQUISITION  MANAGEMENT  SYSTEM 
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project  manager,  through  the  PEO,  to  the  SAE  was  short.  The  basic  premise 
was  that  defense  acquisition  needed  to  be  streamlined  to  run  in  the  same 
manner  as  a  commercial  venture  (Butrica,  2001,  pp.  212-213). 

Another  feature  of  the  acquisition  management  system  is  that  it 
classifies  programs  for  higher  levels  of  oversight  based  upon  expected 
development  or  production  expenditures.  An  Acquisition  Category  I  (ACAT 
I)  Major  Defense  Acquisition  Program  (MDAP),  requiring  oversight  by  the 
Defense  Acquisition  Executive  (DAE)  or  DoD  Component  Acquisition 
Executive  (CAE),  if  so  delegated,  is  a  program  that  is  expected  to  require 
in  excess  of  $365  million  of  Research,  Development,  Test,  and  Evaluation 
(RDT&E)  funds  and/or  $2.19  billion  of  procurement  funds  (in  fiscal  year 
2000  constant  dollars)  (DoD,  2008a,  end.  3,  p.  33). 

Unlike  the  PPBE  process  that  is  calendar-driven  or  the  JCIDS  which 
is  needs-driven,  the  acquisition  management  system  is  event-driven.  All 
acquisition  programs  are  managed  through  a  series  of  sequential  phases 
and  milestone  reviews  (Figure  2).  To  successfully  move  from  one  phase  to 
the  next,  a  program  must  have  demonstrated  or  completed  the  program- 
specific  exit  criteria  for  the  current  phase  and  must  also  have  met  the 
statutory  and  regulatory  entrance  criteria  for  the  next  phase.  The  appointed 
Milestone  Decision  Authority  (MDA)  makes  the  “go/no-go”  decision  based 
on  the  evidence  presented  at  the  milestone  review. 

The  effect  of  having  a  higher  level  decision  maker  for  MDAPs  is  that 
31  percent  of  the  department’s  programmed  Research,  Development,  and 
Acquisition  (RDA)  funds  are  under  the  authority  of  one  decision  maker— the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics,  who 
is  the  designated  DAE.  Yet,  the  remaining  69  percent  of  programmed  RDA 
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funds  are  under  the  control  of  the  Services  and  Defense  Agencies.^  The 
total  number  of  decision  makers  with  MDA  for  lower  priority  acquisition 
programs  is  over  40T 

In  addition,  analysis  of  acquisition  decision  memoranda  (ADMs) 
documenting  the  decisions  of  the  DAE  for  MDAPs  reveals  that  36  percent 
of  the  ADMs  contained  language  with  impact  on  the  requirements  decision¬ 
making  process,  and  66  percent  of  the  ADMs  contained  actions  affecting 
decisions  in  the  budgeting  process."*  Obviously,  decisions  made  on  the 
more  numerous  lower  acquisition  category  programs  also  ripple  into  the 
requirements  and  budgeting  processes  at  higher  rates. 


JCIDS:  CENTRALIZING  THE  VALIDATION  OF  CAPABILITY  DOCUMENTS  TO 
ENSURE  "JOINTNESS" 

Historically,  the  military  services  have  had  their  own  systems  for  the 
approval  of  weapons  systems  requirements.  However,  in  1976  the  Office  of 
Federal  Procurement  Policy  published  Circular  A-109  that  required  a  Mission 
Area  Analysis  to  determine  the  need  for  a  particular  weapons  system  (OMB, 
1976).  In  compliance  with  A-109,  the  Services  were  required  to  perform  this 
analysis  and  prepare  a  mission  needs  statement  to  document  the  need 
at  the  front  end  of  the  acquisition  process  (Fox,  1988,  p.  46).  Eventually, 
to  ensure  that  requirements  were  not  duplicated  between  the  Services 
and  to  prompt  interoperability  and  joint  operations,  the  Joint  Staff  got 
involved.  In  the  early  1990s,  they  required  the  Services  to  adopt  a  single 
document  format  for  the  Operational  Requirements  Document  (ORD).  In 
2003,  the  Joint  Capabilities  Integration  and  Development  System  (JCIDS) 
process  was  created  to  identify  the  capabilities  and  associated  operational 
performance  criteria  required  by  the  joint  warfighter.  JCIDS  also  supports 
the  statutory  responsibility  of  the  Joint  Requirements  Oversight  Council 
(JROC)  to  validate  joint  warfighting  requirements. 

Fundamental  to  JCIDS  is  Capabilities-Based  Assessment  (CBA)  (Figure 
3).  Unlike  the  more  predictable  threats  of  the  cold  war  that  the  Pentagon 
could  anticipate  and  prepare  for,  threats  today  emerge  on  a  daily  basis, 
and  are  often  asymmetrical  to  our  existing  capabilities.  CBA  seeks  to  find 
solutions  to  these  emerging  threats  by  changing  Doctrine,  Organization, 
Training,  Material,  Leadership  and  Education,  Personnel,  and  Facilities 
(DOTMLPF)  (Chairman  of  the  Joint  Chiefs  of  Staff  [CJCS],  2009,  p.  GL-3). 
The  CBA  process  produces  initial  capability,  capability  development,  and 
capability  production  documents  (ICD,  CDD,  and  CPD).  These  documents 
guide  the  technology  development,  engineering  and  manufacturing 
development,  and  production  and  deployment  phases  of  the  acquisition 
framework,  respectively  (Figure  2). 
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FIGURE  3.  CAPABILITIES-BASED  ASSESSMENT 


Existing 

Guidance 


Chairman  of  the  Joint  Chiefs  of  Staff  Instruction  (CJCSI)  3170. OIG 
is  explicit  regarding  how  JCIDS  interfaces  with  the  two  other  decision 
support  systems: 

The  JCIDS  process  supports  the  acquisition  process  by  identifying 
and  assessing  capability  needs  and  associated  performance  criteria 
to  be  used  as  a  basis  for  acquiring  the  right  capabilities,  including 
the  right  systems.  These  capability  needs  then  serve  as  the  basis 
for  the  development  and  production  of  systems  to  fill  those  needs. 
Additionally,  it  provides  the  PPBE  process  with  affordability  advice 
by  assessing  the  development  and  production  life-cycle  cost. 
(CJCS,  2009,  pp.  A-1,  A-2) 

An  approved  ICD  summarizes  the  CBA  process,  describes  the  capability 
gaps,  and  identifies  potential  solutions.  The  ICD  is  taken  to  a  Materiel 
Development  Decision  (MDD)  where  it  is  reviewed  and  validated  in  order 
to  start  the  acquisition  process.  A  favorable  MDD  leads  into  the  Materiel 
Solution  Analysis  phase,  which  is  prior  to  Milestone  A.  In  this  phase,  an 
Analysis  of  Alternatives  (AoA)  is  prepared,  based  upon  the  broad  type  of 
materiel  solution  preferred  in  the  ICD  (i.e.,  information  system,  evolutionary 
development  of  an  existing  capability,  or  a  transformational  approach) 
(CJCS,  2009,  p.  A-3).  Each  alternative  has  an  associated  life-cycle  cost 
that  gives  insight  into  the  affordability  of  the  program  and  provides 
linkage  to  the  budgeting  process.  So,  it  is  important  to  note  that  the 
information  in  the  ICD  drives  the  AoA  process.  The  ICD  also  informs  the 
technology  development  strategy,  the  test  and  evaluation  strategy,  and  the 
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FIGURE  4.  PLANNING,  PROGRAMMING,  BUDGETING,  AND  EXECUTION 
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systems  engineering  plan— all  key  documents  for  guiding  the  technology 
development  phase  prior  to  program  initiation  at  Milestone  B. 


PPBE:  THEN  AND  NOW 

In  the  spring  of  2008,  the  American  Society  of  Military  Comptrollers 
(ASMC)  surveyed  575  members  of  the  defense  financial  management 
community  about  the  PPBE  process  (Figure  4).  Agreement  was  almost 
universal  that  PPBE  was  the  best  method  to  link  performance  and 
budgeting,  “and  a  strong  sentiment  to  fully  implement  the  system  as 
designed”  (ASMC  &  Grant  Thornton  LLP,  2008,  p.  1).  So,  just  what  was 
PPBE  originally  designed  to  do?  And,  has  the  DoD  implemented  PPBE  in 
a  way  that  allows  it  to  do  what  it  was  designed  to  do?  To  find  answers  to 
these  questions,  one  must  go  back  to  the  beginnings  of  PPBE  (then  PPBS) 
during  the  era  of  Defense  Secretary  Robert  S.  McNamara. 

In  1961,  President  Kennedy’s  initial  instructions  to  McNamara  were  “to 
determine  what  forces  were  required  and  to  procure  and  support  them 
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as  economically  as  possible”  (McNamara,  1964,  p.  14).  Developed  by  cost 
analysts  at  the  RAND  Corporation  during  the  1950s,  program  budgeting 
was  just  what  the  Pentagon  needed  to  link  budget  inputs  to  capability 
outcomes  and  to  centralize  long-range  planning  and  financial  decision 
making  under  the  civilian  Secretary  of  Defense.  The  system  was  originally 
called  the  Planning,  Programming,  and  Budgeting  System  (PPBS),  and 
its  fundamental  purpose  was  to  unify  annual  budgets  and  nonfinancial 
longer  range  planning.  In  the  age  of  the  nuclear  bomb,  the  task  of  long- 
range  planning  was  to  calculate  the  needed  effects  or  outputs  that  had  to 
be  produced  by  military  forces  and  weapons  systems  in  order  to  prevail. 
Budgeted  funds  for  these  military  forces  and  weapons  systems  came 
from  the  funding  appropriations  for  military  personnel,  research  and 
development,  procurement,  and  operations  and  maintenance.  Yet,  budgets 
are  resource  inputs.  Moreover,  because  of  the  long  development  cycles  for 
modern  weapons  systems,  annual  budgeting  was  not  a  useful  planning  tool. 
The  key  for  McNamara,  and  the  objective  of  PPBS,  was  to  link  the  planning 
outputs  to  the  appropriated  funds  inputs  through  the  construct  of  defined 
program  elements  within  a  5-year  force  structure  and  financial  program 
(Novick,  1962,  p.  2). 

As  originally  envisioned,  planning  within  the  PPBS  was  to  be  a 
comparative  analysis  of  the  projected  costs  and  effectiveness  of  feasible 
alternatives.  The  example  used  by  David  Novick,  one  of  the  developers  of 
program  budgeting,  is  the  comparison  of  the  merits  of  buying  more  Polaris 
submarines  versus  Minuteman  missile  squadrons.  Both  systems  could  deliver 
nuclear  warheads.  The  comparison  between  the  two  alternatives  involved 
the  methodical  examination  of  the  cost  estimates  for  manpower,  equipment, 
and  facilities,  and  the  expected  military  benefits  (capability  outcomes) 
derived  from  the  systems  (Novick,  1962,  p.  6).  Today,  comparatively  little 
analysis  to  this  level  of  detail  takes  place  in  the  planning  phase  of  PPBE. 
Up  until  2006,  planning  was  simply  an  effort  to  turn  the  National  Security 
Strategy  (NSS),  National  Defense  Strategy  (NDS),  National  Military  Strategy 
(NMS),  and  the  Quadrennial  Defense  Review  (QDR)  into  guidance  from 
which  the  Services  could  develop  their  Program  Objective  Memoranda 
(POMs).  Such  a  shallow  planning  effort  resulted  in  guidance  that  was  not 
specific  enough,  in  terms  of  priorities  and  quantities,  for  the  programming 
of  adequate  resources  for  weapons  systems  acquisitions.  Here  is  but  one 
of  many  examples. 

The  National  Security  Strategy  (JZWritori,  2000)  was  silent  on  the  role  of 
the  military  in  finding  and  taking  the  fight  to  terrorists.  While  the  document 
discusses  the  need  for  the  military  to  help  deter  terrorism  and  respond 
in  retribution  to  terrorist  attacks,  the  mission  of  finding  and  destroying 
terrorist  organizations  is  not  mentioned.  Thus,  the  FYDP  for  fiscal  years 
2002-2007,  prepared  by  the  Pentagon  in  fiscal  year  2000,  lacked  a  vision  for 
the  weapons  systems  and  equipment  necessary  to  prosecute  an  offensive 
global  war  on  terror  (Paparone,  2008,  p.  157).^  As  the  world  changes  at  an 
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unprecedented  pace,  casting  a  meaningful  strategic  vision  becomes  more 
and  more  problematic.  Without  meaningful  strategic  vision,  the  acquisition 
management  system  may  continue  to  acquire  programs  that  will  no  longer 
be  needed— and  may  fail  to  start  programs  that  will  be  needed.  The  Obama 
Administration  has  yet  to  set  clear  national  security  priorities.  As  a  result, 
the  Pentagon  began  in  early  2009  the  planning  phase  for  fiscal  years  2012- 
2017  without  the  benefit  of  an  NSS.  Clearly,  no  one  knows  what  the  future 
will  hold.  However,  planning  for  a  future  we  cannot  see  and  attempting 
to  bring  that  illusion  to  the  future  fight,  with  all  the  associated  weapons 
systems  acquisition  requirements,  is  clearly  folly  if  not  patently  dangerous. 
Yet,  this  is  the  current  planning  process  upon  which  the  Pentagon  justifies 
and  builds  its  6-year  defense  program. 

The  Government  Performance  and  Results  Act  (GPRA)  of  1993  requires 
that  each  government  agency  establish  a  results-oriented  management 
approach  to  strategically  allocate  resources  on  the  basis  of  performance 
(GPRA,  1993).  In  assessing  the  implementation  of  GPRA,  the  Government 
Accountability  Office  has  criticized  the  DoD  for  not  establishing  goals  or 
timelines  for  accountability  and  for  the  measurement  of  progress  toward 
implementation.  DoD  implemented  a  risk  management  framework  in  its 
strategic  plan— the  2001  QDR  report  (GAO,  2005,  p.  8).  However,  it  was 
not  until  2003  that  the  DoD  adopted  the  balanced  scorecard  approach 
to  implement  risk  management.  The  GAO  criticized  the  DoD  for  not 
integrating  this  framework  with  other  decision-making  support  processes. 
Specifically,  the  GAO  said  that  to  be  effective,  risk-based  and  results- 
oriented  management  approaches  have  to  be  integrated  into  the  usual 
cycle  of  agency  decision  making.  The  GAO  presumed  that  without  this  level 
of  integration,  a  mismatch  between  programs  and  budgets  would  continue, 
and  a  proportional  rather  than  strategic  allocation  of  resources  would  go 
to  the  Services.*^  In  addition,  the  Congress  would  not  have  insight  as  to 
the  risks  and  trade-offs  made  during  the  Pentagon’s  investment  decision 
making  (GAO,  2005,  p.  5). 

Chartered  to  examine  how  DoD  develops,  resources,  and  provides  joint 
capabilities,  the  Joint  Capabilities  Study  Team  (also  called  the  Aldridge 
Study)  reported  these  findings  to  Defense  Secretary  Rumsfeld  in  2004: 
“Services  dominate  the  current  requirements  process...;  Service  planning 
does  not  consider  the  full  range  of  solutions  available  to  meet  joint 
warfighting  needs...;  and,  the  resourcing  function  focuses  senior  leadership 
effort  on  fixing  problems  at  the  end  of  the  process,  rather  than  being 
involved  early  in  the  planning  process.”  They  also  found  that  programming 
guidance  exceeds  available  resources  (DoD,  2004,  p.iii).  Others  have  also 
identified  this  programming  guidance  “gap”  (Christie,  2008,  p.  196;  Church 
&  Warner,  2009,  p.  82;  Johnson,  2003,  p.  9). 

The  Aldridge  Study  proposed  a  four-step  process;  strategy,  enhanced 
planning,  resourcing,  and  execution  and  accountability.  The  strategy  step 
involved  the  combatant  commanders  and  answered  the  question:  “What  to 
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do?”  The  enhanced  planning  and  resourcing  steps  answered  the  question: 
“How  to  do  it?”  The  execution  and  accountability  step  answered  the 
question:  “How  well  did  we  do?”  Formal  process  review  points  for  the 
Secretary  of  Defense  were  proposed  after  each  of  the  four  steps  (DoD, 
2004,  p.  V). 

Many  of  the  recommendations  from  the  Aldridge  Study  were 
implemented.  Most  notably,  the  Enhanced  Planning  Process  (EPP)  was 
made  a  phase  of  the  Strategic  Planning  Process,  and  the  EPP  is  to  be 
approved  by  the  Secretary  or  Deputy  Secretary  of  Defense.  Moreover,  the 
Joint  Programming  Guidance  is  to  document  the  decisions  resulting  from 
the  EPP  phase  (DoD,  2006,  p.  2).  The  Director  of  Program  Analysis  and 
Evaluation  (PA&E)  already  had  responsibility  as  the  Office  of  the  Secretary 
of  Defense  (OSD)  lead  for  coordinating  the  program  review  of  the  PPBE 
process.  The  only  problem  with  this  new  assignment  is  that  it  appears  to 
conflict  with  the  responsibilities  of  the  Under  Secretary  of  Defense  (Policy), 
who  has  overall  responsibility  for  coordinating  the  PPBE  planning  phase 
(DoD,  2003,  p.5). 

Another  problem  for  PPBE  is  that  developing  and  finally  enacting  the 
first  year  of  the  6-year  program  takes  a  long  time.  The  program  (termed 
Future  Years  Defense  Program  or  FYDP)  is  put  together  only  once  every 
2  years,  during  even  numbered  years.  For  example,  in  calendar  year  2010, 
the  Pentagon  will  put  together  the  6-year  program  for  fiscal  years  (FY) 
2012  through  2017.  However,  the  Services  began  working  on  their  portions 
of  that  FY2012-2017  program  in  the  middle  of  calendar  year  2009— more 
than  3  years  before  the  first  year  funds  for  FY  2012  will  be  appropriated 
by  the  Congress.  The  next  opportunity  to  make  major  changes  to  the 
program  is  in  calendar  year  2012  when  the  program  for  FY  2014-2019  will 
be  accepted  by  the  Pentagon.  Changes  to  the  program  are  possible  during 
the  odd  numbered  years.  However,  these  changes  are  usually  limited  to 
necessary  fact-of-life  adjustments.  New  starts  (or  stops)  are  generally  not 
considered  in  the  odd  numbered  years.  Thus,  the  programming  phase  of  the 
PPBE  process  suffers  from  false  precision.  Even  if  the  vision  of  the  future 
was  correctly  identified  in  the  planning  phase,  programming  for  weapons 
systems  new  starts  can  only  be  done  every  other  year.  Moreover,  funds 
requested  are  for  use  more  than  three  or  more  years  hence.  Inevitably, 
projections  for  weapons  systems  costs  that  far  in  advance  of  execution  are 
bound  to  be  flawed.  Yet,  the  process  demands  precision,  whether  or  not 
that  precision  has  any  meaning  (McCaffery  &  Jones,  2005,  p.  159). 

As  originally  envisioned.  Secretary  McNamara  expected  to  conduct 
a  continuous  review  of  the  entire  defense  program.  In  other  words,  he 
expected  to  have  an  up-to-date  5-year  force  structure  and  financial 
program  at  all  times.  McNamara’s  PPBS  had  a  program  change  control 
system  in  which  variations  from  approved  cost  estimates  required 
advance  authorization.  Standard  forms  were  established  for  research  and 
development,  investment,  and  operations— each  relating  to  the  key  decision 
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points  in  the  life  of  a  weapon  system.  The  program  change  control  system 
was  first  applied  to  200  of  the  most  important  material  systems.  Milestone 
schedules  were  prepared  for  these  systems,  and  actual  progress  was 
reported  on  a  monthly  basis,  including  the  need  for  corrective  action  or 
revision  to  the  financial  plan  (Novick,  1962,  pp.  7-10).  Such  is  not  the  case 
today.  The  FYDP  is  open  for  changes  only  twice  a  year— in  August  when 
ROMs  (or  changes  to  the  previous  ROMs)  are  submitted  by  the  Services 
to  OSD,  and  at  the  end  of  the  combined  program  and  budget  review  once 
resource  management  decisions  have  been  made  and  the  defense  budget 
is  finalized  for  the  Office  of  Management  and  Budget. 

In  his  first  year  as  Defense  Secretary,  McNamara  was  heavily  involved 
in  the  cost-effectiveness  and  requirements  studies  of  the  planning  phase 
of  RRBS.  Known  as  “McNamara’s  100  Trombones,”  he  assigned  about  100 
requirements  projects  to  the  Joint  Chiefs  of  Staff,  the  Services,  and  various 
elements  of  OSD.  These  planning  studies  were  truly  participative  in  nature 
and  required  a  significant  time  commitment  from  McNamara,  but  they 
resulted  in  detailed  acquisition  programming  guidance  for  the  Services. 
For  example,  in  his  first  year  McNamara  made  decisions  on  the  number  of 
strategic  missiles  and  bombers  for  the  next  decade.  Fie  also  decided  on  the 
airlift  and  sealift  needed  to  support  contingency  war  plans  and  the  most 
cost-effective  way  of  replacing  worn  out  ground  equipment  for  the  Army 
(Hitch,  1965,  pp.  74-75). 

Senior  leader  involvement  in  today’s  PPBE  process  has  typically 
been  toward  the  end  of  the  programming  phase  rather  than  in  the  earlier 
planning  phase.  This  is  not  the  optimum  time  for  these  senior  leaders  to 
enter  the  PPBE  decision-making  process.  Moreover,  failing  to  make  the 
tough  decisions  up  front  in  the  planning  phase  only  delays  them  into  late 
in  the  programming  phase  (Johnson,  2003,  pp.  10-11).  Decisions  become 
harder  to  make  during  the  final  stages  of  programming  because  less 
discretionary  funding  is  available,  and  earlier  decisions  will  need  to  be 
reconsidered.  Such  late  decision  making  on  weapons  systems  acquisition 
terminations  was  typical  in  past  PPBE  cycles.  However,  as  demonstrated 
by  Defense  Secretary  Gates  during  the  2010  budget  deliberations,  he  may 
get  more  involved  up  front  and  make  these  types  of  decisions  early  in  the 
planning  phase  of  PPBE. 

Today,  PPBE  fiscal  and  programming  guidance  is  usually  late  in  arriving 
to  the  Services.  While  no  directive  or  instruction  establishes  a  date  for 
issuance  of  fiscal/programming  guidance,  issuance  dates  for  the  past  two 
PPBE  cycles  were  March  14,  2008,  for  POM  10-15;  and  May  7,  2009,  for 
POM  11-15.  Fiscal  guidance  refers  to  the  total  obligation  authority,  by  fiscal 
year,  available  to  the  Services.  Fiscal/programming  guidance  is  used  by  the 
Services  to  develop  their  ROMs,  or  changes  to  the  previous  POM,  which  are 
usually  due  in  August.  They  begin  development  of  their  ROMs  in  the  last  few 
months  of  the  prior  year  (October-December  timeframe).  While  draft  fiscal/ 
programming  guidance  is  often  released  earlier,  final  fiscal/programming 
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guidance  is  usually  issued  too  late  to  be  useful.  Today,  fiscal/programming 
guidance  is  found  in  the  “fiscally  informed”  Guidance  for  the  Development 
of  the  Force  (GDF)  and  the  “fiscally  constrained”  Joint  Programming 
Guidance  (JPG)  (Church  &  Warner,  2009,  p.  84).  The  predecessor  to  the 
GDF  was  the  Strategic  Planning  Guidance  (SPG),  and  before  the  SPG,  the 
Defense  Planning  Guidance  (DPG).  Originally  envisioned  to  align  strategy 
with  investments,  the  GDF  appears  to  have  become  a  “wish  list  of  programs 
and  priorities  for  every  constituency.”  Feedback  from  the  Services  on  the 
usefulness  of  the  GDF  and  JPG  is  mixed.  As  indicated,  both  documents, 
but  especially  the  JPG,  are  issued  well  after  the  Services  have  completed 
the  development  of  their  POMs  and  decisions  made  to  fund  or  not  fund 
various  weapons  systems  programs  (Church  &  Warner,  2009,  pp.  81-82). 

Understandably,  and  working  at  a  disadvantage  with  unclear 
programming  guidance,  the  Service  POMs  are  invariably  criticized  for 
failing  to  comply  with  the  GDF/JPG.  In  addition,  the  POMs  are  faulted 
for  underestimating  technology  risks  associated  with  weapons  systems 
investments  (Christie,  2008,  p.  212).  As  a  result,  the  Services  tend  to 
over  program,  believing  they  can  develop,  produce,  and  place  in  operation 
many  more  programs  than  realistically  possible  (Christie,  2008,  p.  196; 
Church  &  Warner,  2009,  p.  82).  In  other  words,  their  6-year  programs  fail  to 
consider  the  cost  “tails”  past  the  last  year  of  the  FYDP.  This  is  particularly 
a  problem  with  weapons  systems  production  programs  that  build  up  to  an 
unrealistically  high  “bow  wave”  of  procurement  funding  beyond  the  FYDP 
that  becomes  unaffordable  for  the  Service  and  DoD. 

Per  DoD  Directive  7045.14,  the  official  linkage  between  the  PPBE  and 
acquisition  management  systems  is  achieved  by  designated  membership 
on  the  Defense  Systems  Acquisition  Review  Council  (now  the  Defense 
Acquisition  Board  [DAB]),  the  Defense  Resources  Board  (now  the  Deputy’s 
Advisory  Working  Group  [DAWG]),  and  the  Senior  Leader  Review  Group 
(SLRG);  and  the  requirement  to  develop  an  acquisition  strategy  for  all 
major  systems  (DoD,  1984,  reissued  1987,  p.  6).  The  DAB  is  chaired  by  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
who  is  also  a  member  of  the  SLRG  and  DAWG.  The  SLRG  is  chaired  by  the 
Secretary  of  Defense,  and  the  DAWG  is  chaired  by  the  Deputy  Secretary 
of  Defense,  neither  of  whom  sits  on  the  DAB.  In  total,  11  senior  leaders  are 
members  of  both  the  DAB  and  the  SLRG/DAWG.^  The  average  tenure  of 
the  DAE  is  just  24  months.®  Most  MDAPs  have  development  cycles  that 
exceed  the  tenure  of  four  or  even  five  DAEs.  Therefore,  the  effectiveness 
of  having  senior  leaders  serve  as  the  linkage  between  the  resourcing 
and  acquisition  management  systems  might  be  questioned,  given  their 
enormous  responsibilities  and  brief  tenures  serving  as  the  DAE.  Certainly, 
11  senior  leaders  cannot  be  held  responsible  for  coordinating  the  multitude 
of  interactions  between  the  acquisition  and  budgeting  systems. 
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Recommendations 

In  1979,  the  Defense  Resource  Management  Study  (DRMS) 
recommended  to  President  Carter  that  the  programming  and  budgeting 
phases  of  PPBE  be  combined  into  a  single  annual  review.  The  DRMS  also 
recommended  that  the  time  freed  up  by  combining  the  two  phases  be 
used  to  "focus  additional  attention  on  the  strategic  and  resource  planning 
issues,  including  resolution  of  selected  major  issues  prior  to  the  program/ 
budget  review”  (Rice,  1979,  p.  viii).  This  was  the  centerpiece  of  the  DRMS 
proposal,  and  it  was  designed  to  open  up  a  “broad  planning  window” 
that  would  include  "an  orchestrated  OSD  review  and  prioritization  of 
the  Defense  Systems  Acquisition  Review  Council-approved  programs 
competing  for  segments  of  the  planning  wedge”  (Rice,  1979,  pp.  9,  16). 
These  recommendations  were  not  implemented.  However,  in  2003, 
Defense  Secretary  Rumsfeld  did  combine  programming  and  budgeting 
phases,  but  not  with  the  intention  of  freeing  up  time  for  better  planning. 
Rather,  Rumsfeld’s  Management  Initiative  Decision  913  specified  that  the 
freed  up  time  would  be  used  for  an  execution  review  (i.e.,  the  new  “E”  in 
PPBE)  to  “make  assessments  concerning  current  and  previous  resource 
allocations  and  whether  the  department  achieved  its  planned  performance 
goals”  (DoD,  2003,  p.  7;  Church  &  Warner,  2009,  p.  81;  Dawe  &  Jones, 
2005,  p.  49;  Jones  &  McCaffery,  2005,  p.  90).  The  Pentagon  has  yet  to 
institutionalize  this  execution  review.  A  recent  survey  of  575  professionals 
in  the  defense  finance  and  accounting  community  found  that,  due  to  the 
wartime  supplemental  funding  for  operations  in  Iraq  and  Afghanistan, 
emphasis  on  execution  had  not  made  the  relationship  between  budget 
execution  and  performance  more  visible,  nor  had  it  provided  the  data 
needed  to  make  more  timely  decisions  to  improve  the  PPBE  process 
(ASMC  &  Grant  Thornton  LLP,  2008,  pp.  5-7).  Perhaps,  the  “broad  planning 
window”  recommendation  of  the  DRMS  should  again  be  considered,  and 
this  time  implemented,  to  help  resolve  and  clarify  competing  requirements 
and  acquisition  programs  before  the  Services  have  to  prepare  their  ROMs. 

In  2007,  Capability  Portfolio  Management  was  introduced  to  the 
programming  phase  of  PPBE.  The  official  definition  of  Capability  Portfolio 
Management  is  “the  process  of  integrating,  synchronizing,  and  coordinating 
DoD  capabilities  needs  with  current  and  planned  DOTMLPF  investments 
within  a  capability  portfolio  to  better  inform  decision  making  and 
optimize  defense  resources”  (DoD,  2008c,  p.  8).  The  Capability  Portfolio 
Management  initiative  seeks  to  place  all  current  and  proposed  warfighting 
needs  into  logical,  manageable  functional  categories.  In  an  effort  to 
minimize  redundant  capabilities,  capability  portfolios  are  joint,  not  Service- 
specific.  Capability  Portfolio  Managers  (CPMs)  provide  cross-Component 
alternatives  and  recommendations  on  current  and  future  capability  needs 
and  investments.  They  are  to  work  with  the  JROC  and  the  JCIDS,  and 
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develop  capability  planning  guidance  for  inclusion  in  the  GDF.  Therefore, 
CPMs  can  impact  capability  portfolio  composition,  weapons  systems 
acquisition,  and  weapons  systems  sustainment  choices.  In  retrospect,  the 
job  of  the  CPMs  is  similar  to  the  system  analysts  of  the  McNamara  era.  The 
systems  analysts  prepared  “cost-effectiveness  studies”  and  “requirements 
studies”  at  the  request  of  the  Secretary  of  Defense  and  the  Joint  Chiefs  of 
Staff  (Hitch,  1965,  pp.  73-75).  However,  the  advice  of  current  day  CPMs  is 
officially  sought  only  at  the  end  of  the  programming  phase  of  PPBE  when 
they  provide  the  DAWG  with  independent  programmatic  recommendations 
and  cross-Component  perspectives  on  planned  and  proposed  capability 
investments  (DoD,  2008c,  p.  6).  To  have  greatest  influence,  decision  makers 
need  to  formally  tap  into  the  advice  of  these  CPMs  about  9  to  12  months 
earlier,  during  the  planning  phase  of  the  PPBE  process. 

The  deliberate,  evolutionary  pace  of  the  cold  war  is  long  past.  The 
challenges  of  an  ever  changing  global  security  environment  and  the  rapid 
pace  of  technological  advancement  represent  a  national  imperative  for  the 
Pentagon  to  seek  out  and  cultivate  breakthrough  ideas  in  the  development 
and  employment  of  defense  systems  (Johnson,  2003,  pp.  6-7).  To  meet 
these  challenges,  the  PPBE  planning  phase  should  be  revitalized  and 
extended  to  allow  time  for  brainstorming  and  germination  of  innovative 
ideas,  and  for  the  analysis  of  the  costs  and  effectiveness  of  various  weapons 
systems  alternatives. 

Conclusions 

As  implemented  today,  the  PPBE  process  is  far  different  from  the  PPBS 
established  by  Secretary  of  Defense  McNamara  in  1961.  Over  the  course  of 
nearly  50  years,  changes  have  severely  de-emphasized  decision  making 
in  the  planning  phase.  As  a  result,  the  department  has  had  to  establish  a 
separate  requirements  analysis  and  approval  system.  The  concept  behind 
today's  JCIDS  was  actually  part  of  McNamara's  long-range  planning 
to  determine  the  most  cost-effective  capability  outcomes.  Likewise,  in 
McNamara’s  management  system,  weapons  systems  development  and 
production  decisions,  along  with  necessary  funding  adjustments,  were 
made  in  real  time,  and  at  the  same  time  as  requirements  decisions.  Today, 
the  linkage  between  PPBE  and  weapons  systems  decisions  suffers  from 
the  timing  disconnect  between  a  calendar-driven  budget  and  event-driven 
acquisition  programs.  To  improve  acquisition  decision  making,  the  linkages 
between  the  requirements,  budgeting,  and  acquisition  decision-making 
systems  must  be  reestablished.  One  solution  is  to  reinvigorate  the  planning 
phase  of  PPBE  and  make  the  necessary  decisions  on  weapons  systems 
requirements,  multiyear  budgeting,  and  acquisition  program  continuation 
or  termination,  within  the  timeframe  of  that  phase. 
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ENDNOTES 

1.  This  problem  has  not  been  fixed.  Writing  in  the  January/February  2009  issue  of  Foreign 
Affairs,  Secretary  of  Defense  Robert  M.  Gates  (2009)  called  for  a  reassessment  of 
priorities  within  the  Department  of  Defense; 

The  defining  principle  of  the  Pentagon's  new  National  Defense  Strategy  is  balance. 
The  United  States  cannot  expect  to  eliminate  national  security  risks  through  higher 
defense  budgets,  to  do  everything  and  buy  everything.  The  Department  of  Defense 
must  set  priorities  and  consider  inescapable  tradeoffs  and  opportunity  costs. 

The  strategy  strives  for  balance  in  three  areas:  between  trying  to  prevail  in 
current  conflicts  and  preparing  for  other  contingencies,  between  institutional¬ 
izing  capabilities  such  as  counterinsurgency  and  foreign  military  assistance  and 
maintaining  the  United  States'  existing  conventional  and  strategic  technological 
edge  against  other  military  forces,  and  between  retaining  those  cultural  traits 
that  have  made  the  U.S.  armed  forces  successful  and  shedding  those  that  ham¬ 
per  their  ability  to  do  what  needs  to  be  done.  (p.  28). 

How  Gates  will  achieve  this  rebalancing  of  priorities  is  the  essence  of  this  research. 

2.  In  Future  Years  Defense  Program  2008-2013  (FYDP  2008-2013),  the  total  obligation 
authority  for  RDT&E  and  Procurement  was  $1,154  billion.  By  virtue  of  the  fact  that  the 
Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics  is  the  MDA  for 
MDAPs,  the  OSD  has  control  over  acquisition  decisions  totaling  $362  billion,  or  about  31 
percent  of  the  total  obligation  authority  in  FYDP  08-13.  On  the  other  hand,  the  Services 
make  decisions  on  about  $792  billion,  or  about  69  percent  of  the  total  obligation 
authority  for  RDT&E  and  procurement  in  FYDP  08-13  (DoD,  2008d,  Table  1-9,  p.  13; 
DAMIR,  n.d.,  MDAP/MAIS  Selected  Acquisition  Report  query,  FYDP  08-13). 

3.  Each  Service  and  Defense  Agency  has  an  Acquisition  Executive  (AE)  with  MDA.  In 
addition,  all  PEOs  have  MDA.  The  total  number  of  PEOs  is  35  (Army-11;  Navy-13;  Air 
Force-11).  (Source:  Organizational  charts  of  Army,  Navy,  and  Air  Force  AEs.  Retrieved 
November  14,  2009,  from  https://www.alt.army.mil/portal/page/portal/oasaalt, 
https;//acquisition. navy.mil/rda/content/view/fu 1 1/4539,  http://ww3.safaq.hq.af.mil/ 
organizations/index. asp) 

4.  The  Defense  Acquisition  Management  System  (DAMS)  uses  ADMs  as  records  of  the 
decision  made  by  the  AE.  For  purposes  of  this  research,  ADMs  for  the  following  weapons 
systems  were  reviewed:  Expeditionary  Fighting  Vehicle  (6  ADMs);  Future  Combat 
System  (10  ADMs);  Global  Hawk  (12  ADMs);  Joint  Strike  Fighter  (13  ADMs);  and  Littoral 
Combat  Ship  (3  ADMs).  In  total,  44  ADMs  were  reviewed.  Of  these,  36  percent  (16  ADMs) 
contained  actions  that  would  require  involvement  of  the  JCIDS.  In  addition,  66  percent 
(29  ADMs)  contained  actions  that  would  impact  upon  the  PPBE  process  (ADM,  n.d.). 

5.  COL  Christopher  R.  Paparone,  USA  (Ret.),  makes  an  argument  that  the  Joint  Vision  2020, 
published  in  June  2000,  focused  on  defensive  force  protection  from  terrorists,  not  on 
the  use  of  military  forces  to  combat  terrorism  in  an  offensive  way,  which  was  the  case 
after  September  11,  2001.  While  the  Joint  Vision  2020  was  not  a  PPBE  document,  per 

se,  his  point  is  applicable.  Combating  terrorists  offensively  is  not  seen  in  the  National 
Security  Strategy  prior  to  9/11.  This  is  not  the  only  failure  on  the  part  of  past  Presidential 
Administrations  in  providing  meaningful  strategic  priorities.  Ihe  2002  Nationai  Security 
Strategy  (NSS)  failed  to  envision  the  invasion  of  Iraq  on  March  20,  2003,  the  fall  of 
Baghdad,  and  the  associated  requirements  for  nation  building  that  were  thrust  onto 
the  military.  The  2004  National  Military  Strategy  (NMS)  failed  to  envision  the  need 
for  massive  humanitarian  aid  in  the  wake  of  the  Indian  Ocean  tsunami  of  December 
24,  2004,  and  the  associated  requirements  that  the  military  would  need  for  logistical 
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support  across  the  shores  of  devastated  islands  and  coastal  regions.  Similarly,  the  2005 
National  Defense  Strategy  (NDS)  failed  to  envision  that  North  Korea  would  test  fire 
missiles  over  the  Sea  of  Japan  on  July  4,  2006,  and  subsequently  explode  a  nuclear 
device  in  the  mountains  on  October  9,  2006.  The  2005  NDS  makes  no  mention  of  our 
nation’s  need  to  acquire  an  integrated  missile  defense  capability. 

6.  The  Government  Accountability  Office  says  that  even  though  the  DoD  has  adopted  a  risk 
management  planning  framework  and  balanced  scorecard  approach  to  programming 
for  outcomes,  the  percentage  of  total  obligation  authority  in  the  FYDP,  by  Service, 
has  remained  relatively  unchanged.  The  GAO  provided  the  following  figures  in  its 
report.  Defense  Management:  Additional  Actions  Needed  to  Enhance  DoD’s  Risk-based 
Approach  for  Making  Resource  Decisions  (GAO-06-13): 


TABLE.  MILITARY  SERVICE  AND  DEFENSE-WIDE  PERCENTAGE 
OF  THE  2005  AND  2006  FUTURE  YEARS  DEFENSE  PROGRAMS 


2005 

2006 

Percentage 

Percentage 

Percentage 

Change  by 

of  FYDP 

of  FYDP 

Department 

Department  of 

24.23 

24.63 

0.40 

the  Army 

Department  of 

29.75 

29.47 

-0.28 

the  Navy 

Department  of 

29.80 

29.82 

0.02 

the  Air  Force 

Defense-wide 

16.22 

16.08 

-0.14 

Total 

100.00 

100.00 

(Source:  GAO,  2005,  p.  16) 


7.  The  members  of  both  the  DAB  and  the  SLRG/DAWG  are:  Under  Secretary  of  Defense 
for  Acquisition,  Technology  and  Logistics;  Vice  Chairman  of  the  Joint  Chiefs  of  Staff; 
Secretaries  of  the  Military  Departments;  Under  Secretary  of  Defense  (Policy);  Under 
Secretary  of  Defense  (Comptroller);  Under  Secretary  of  Defense  (Personnel  and 
Readiness);  Under  Secretary  of  Defense  (Intelligence);  Assistant  Secretary  of  Defense 
for  Networks  and  Information  Integration/DoD  Chief  Information  Officer;  Director,  Cost 
Assessment  and  Program  Evaluation  (DoD,  2008b,  ends.  3  &  4;  DoD,  2009,  p.  10.2.1). 

8.  From  Richard  Godwin,  the  first  Under  Secretary  of  Defense  (Acquisition)  until  Ashton 
Carter,  the  current  Under  Secretary  of  Defense  for  Acquisition,  Technology  and  Logistics, 
average  tenure  has  been  24  months.  To  date,  the  shortest  service  was  by  Godwin  who 
served  12  months  (September  1986-September  1987),  and  the  longest  service  was  by 
Jacques  Gansler,  who  served  38  months  (November  1997-January  2001)  (Brown,  2005). 
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ICD 
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JROC 
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APPENDIX 

List  of  Abbreviations  and  Acronyms 
Acquisition  Category 

Acquisition  Decision  Memorandum  or  Acquisition  Decision 

Memoranda 

Acquisition  Executive 

Analysis  of  Alternatives 

American  Society  of  Military  Comptrollers 

Budget  Estimate  Submission 

Component  Acquisition  Executive 

Capabilities-Based  Assessment 

Capability  Development  Document 

Critical  Design  Review 

Chairman  of  the  Joint  Chiefs  of  Staff 

Chairman  of  the  Joint  Chiefs  of  Staff  Instruction 

Capability  Portfolio  Manager 

Defense  Acquisition  Board 

Defense  Acquisition  Executive 

Defense  Acquisition  Management  System 

Defense  Acquisition  Performance  Assessment 

Deputy’s  Advisory  Working  Group 

Department  of  Defense 

Department  of  Defense  Directive 

Department  of  Defense  Instruction 

Doctrine,  Organization,  Training,  Material,  Leadership  and 

Education,  Personnel,  and  Facilities 

Defense  Planning  Guidance 

Defense  Resource  Management  Study 

enclosure 

Enhanced  Planning  Process 
Full  Rate  Production 
Fiscal  Year 

Future  Years  Defense  Program 
Government  Accountability  Office 
Guidance  for  the  Development  of  the  Force 
Government  Performance  and  Results  Act 
Initial  Capabilities  Document 

Joint  Capabilities  Integration  and  Development  System 

Joint  Planning  Guidance 

Joint  Requirements  Oversight  Council 

Joint  Staff 

Milestone  Decision  Authority 
Major  Defense  Acquisition  Program 
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NDS 

NMS 

NSS 

OMB 

ORD 

OSD 

PA&E 

PBD 

PDM 

PDR 

PEO 

POM 

PPBE 

PPBS 

QDR 

RDA 

RDT&E 

SAE 

SLRG 

SPG 


National  Defense  Strategy 
National  Military  Strategy 
National  Security  Strategy 
Office  of  Management  and  Budget 
Operational  Requirements  Document 
Office  of  the  Secretary  of  Defense 
Program  Analysis  and  Evaluation 
Program  Budget  Decision 
Program  Decision  Memorandum 
Preliminary  Design  Review 

Program  Executive  Office  or  Program  Executive  Officer 
Program  Objective  Memorandum  or  Program  Objective 
Memoranda 

Planning,  Programming,  Budget,  and  Execution 
Planning,  Programming,  and  Budgeting  System 
Quadrennial  Defense  Review 
Research,  Development,  and  Acquisition 
Research,  Development,  Test,  and  Evaluation 
Service  Acquisition  Executive 
Senior  Leader  Review  Group 
Strategic  Planning  Guidance 
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IT’S  TIME  TO  TAKE 
THE  CHILL  OUT 
OF  COST  CONTAINMENT 
AND  RE-ENERGIZE  A  KEY 
ACQUISITION  PRACTICE 

f  Co!  Robert  L.  Tremaine,  U SAP  (Ret.) 
and  Donna  J.  Seligman 

Unless  program  managers  (PM)  tackle  cost  containment 
head-on,  future  weapon  system  acquisition  successes  may 
be  jeopardized,  resulting  in  fewer  products  and  services  to 
equip  the  nation’s  warfighters.  The  United  States  can  ill  afford 
any  decrease  in  its  preparedness  when  the  nation  is  currently 
waging  war  on  two  fronts.  This  research  examines  cost  contain¬ 
ment  in  the  context  of  Total  Life  Cycle  Cost  Management.  A 
more  thorough  understanding  and  aggressive  application  of 
cost-containment  strategies  could  conceivably  shift  acquisition 
outcomes  to  a  more  cost-effective  posture.  Responding  to  a 
survey  conducted  as  part  of  this  research,  887  Department  of 
Defense  (DoD)  acquisition  professionals  provided  Input  on  cost 
containment,  including  tool  types  and  associated  processes.  Of 
those  887  respondents,  543  were  current  or  former  DoD  PMs. 


Keywords:  Life  Cycle  Cost  Management  (LCCM), 
Cost  Containment,  Cost  as  an  Independent  Variable 
(CAIV),  Performance  Based  Logistics  (PBL), 

Cost  Anaiysis  Requirements  Description  (CARD), 
Earned  Vaiue  Management  (EVM),  Technology 
Readiness  Level  (TRL) 
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Is  there  a  superior  acquisition  development  decision  aid  that  can  assure 
more  program  successes  and  help  contain  costs?  Interestingly  enough, 
some  of  the  most  basic  tools  currently  at  our  disposal  in  the  Department 
of  Defense  (DoD)  are  already  ideally  suited  to  help  achieve  acquisition 
excellence.  They  can  also  have  a  significant  impact  on  fiscal  outcomes.  For 
some  time,  program  managers  (PMs)  have  had  access  to  these  in  the  form 
of  a  customized  Tool  Kit  that  outlines  and  characterizes  a  wide  array  of 
helpful  decision  aids  and  measures  (Defense  Acquisition  University  [DAU], 
2009a),  including: 

•  Technology  Readiness  Level  (TRL).  Tempers  technology 
insertion  by  measuring  technology  maturity;  ensures 
technology  properly  finds  its  way  into  development  efforts, 
while  accounting  for  any  associated  risks;  and  considers 
performance  and  life-cycle  factors  before  a  technology 
solution  is  finalized  (Figure  1) 

FIGURE  TECHNOLOGY  READINESS  LEVEL  SCALE 


(Source:  Labay,  2009) 
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•  Earned  Value  Management  (EVM).  Predicts  cost  and 

schedule  perturbations,  provides  early  warning,  and  serves 
as  a  forecasting  tool  that  ties  itself  to  traceable  physical 
work  packages  (under  an  overall  Work  Breakdown  Structure 
[WBS])  (Figure  2) 
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FIGURE  2.  EARNED  VALUE  MANAGEMENT  (EVM)  GRAPH 


Note.  BCWS=Budgeted  Cost  For  Work  Scheduled;  BCWP=Budgeted  Cost  for  Work 
Performed:  ACWP=Actual  Cost  of  Work  Performed;  EAC=Estimate  At  Completion; 
TAB=Tota/  Allocated  Budget:  BAC=Budget  At  Completion 
(Source:  DAU,  2009b) 


FIGURE  3.  COST  ANALYSIS  REQUIREMENTS  DESCRIPTION 
(CARD) 
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FIGURE  4.  TECHNICAL  AND  MANAGEMENT  PROCESSES 
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Technical  Management  Processes 
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(include  requirements 

Technical  Planning 

development,  logical  analysis, 
and  design  solution) 

Technical  Assessment 

Bottom-Up  Realization 
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Decision  Analysis 

(include  implementation, 
integration,  verification, 
validation,  and  transition) 

Technical  Control  Processes 
(include  requirements  management, 
risk  management,  configuration 
management,  and  technical  data 
management) 

(Source:  Tremaine,  2009) 


•  Cost  Analysis  Requirements  Description  (CARD).  Provides 
comprehensive  and  detailed  descriptions  of  acquisition 
programs;  supports  Program  Office  Estimates  (POE), 

Component  Cost  Analyses  (CCA),  and  independent  Life 
Cycle  Cost  Estimates  (LCCE)  (Figure  3) 

•  Technical  and  Management  Processes.  Ensure  products 
properly  evolve  from  concept  to  deployment;  set  the  stage 
for  the  selection  of  a  wide  range  of  alternative  design 
approaches  through  an  integrated  superset  of  design, 
assessment,  and  control  processes  (Figure  4) 

•  Performance-Based  Logistics  (PBL).  “Provides  a  means 
for  the  resource-constrained  program  management  office 
to  develop,  implement,  and  manage  the  sustainment  of  a 
system  over  its  life  cycle”  (Fowler,  2009)  (Figure  5) 

•  Cost  as  an  Independent  Variable  (CAIV).  Weighs  affordable 
performance  capabilities  and  scheduling  based  on  cost 
goals  that  can  be  realized  by  a  set  of  decisions  that 
balances  programmatic  risks  (Rush,  1997).  Also  serves  as  a 
trade-off  tool  to  achieve  Reduced  Total  Ownership  Costs 
(Pallas  &  Novak,  2000)  (Figure  6). 

Taken  together,  these  tools  can  give  PMs  the  power  to  overcome 
many  of  the  looming  programmatic  hurdles  that  continue  to  surface 
as  often  as  the  weather  changes.  Many  other  helpful  decision  aids  are 
available  and  designed  specifically  to  combat  the  challenges  PMs  face 
every  day.  Considering  this  wide  and  diverse  array  of  decision  aids,  what 
is  missing?  What  have  we  actually  failed  to  characterize  that  ostensibly 
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FIGURE  5.  PERFORMANCE-BASED  LOGISTICS  12-STEP  PROCESS 
MODEL 
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FIGURE  6.  CAIV  FOCUSES  ON  THE  “KNEE  OF  THE  CURVE’ 
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fuels  cost  growth?  Why  do  examples  keep  surfacing  like  the  MV-22  Osprey, 
where  costs  per  flight  hour— currently  at  $11,000— are  expected  to  more 
than  double  the  target  estimate  (Clark,  2009)?  If  so  many  variable  costs 
can  fluctuate,  can  they  be  properly  tracked  and  addressed  in  time  to 
contain  costs? 

One  methodology  in  particular  was  expected  to  give  truthful  predictions 
of  total  costs.  But,  its  value  has  presumably  diminished  in  the  face  of  the 
very  dynamic  and  complex  processes  normally  associated  with  acquisition 
programs  in  the  DoD.  It  goes  by  the  name  Life  Cycle  Cost  Management 
(LCCM).  Up  to  now,  it  has  been  used  to  understand  both  the  wide  array  of 
system  costs  that  start  with  a  program’s  initial  baseline  and  run  all  the  way 
through  disposal. 

Discussion 

Conceptually,  LCCM  is  not  new.  As  early  as  1936,  T.  P.  Wright  had 
already  created  cost  estimating  equations  to  predict  the  cost  of  airplanes 
over  long  production  runs  (Hamaker,  1994).  Oddly  enough,  many  are  still 
in  use  today.  In  varying  degrees,  support  for  LCCM  continued  to  grow  ever 
since.  In  1975,  an  Air  Force  working  group  recommended  five  required 
actions  to  effectively  institute  LCCM  capabilities  in  program  offices.  They 
recommended: 

•  Program  offices  be  provided  with  a  source  of  personnel 
familiar  with  analytical  techniques 

•  Engineers  and  analysts  be  given  general  guidance  on 
how  to  develop,  adapt,  and  use  life-cycle  cost  models  for 
specific  applications 

•  Program  office  and  supporting  personnel  have  access  to  a 
short  course  in  the  subject  of  development  and  application 
of  LCC  models  and  methods 

•  Periodic  life-cycle  cost  methods  workshops  be  held 

•  Program  office  personnel  be  provided  with  a  central  focus 
of  expertise  where  lessons  learned  in  each  new  life-cycle 
application  are  integrated  with  existing  LCC  models  and 
methods  (McKenzie,  1978). 

LCCM  is  certainly  not  an  underdeveloped  concept,  either.  Over  the 
years,  a  number  of  LCC  models  have  surfaced  to  help  programs  fashion  their 
overall  funding  profiles.  Each  model  takes  into  account  the  broad  range  of 
a  system’s  true  costs,  including  its  economic  life,  inflation  rates,  discount 
rates,  total  number  of  cost  elements  that  comprise  the  system,  magnitude 
of  cost  elements,  and  salvage  value,  etc.  But  to  this  day,  when  asked  about 
their  experience  with  LCC  models,  their  applicability,  usefulness,  ease  of 
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TABLE  1.  VALUE  OF  LIFE  CYCLE  COST  MANAGEMENT:  VIEWS  OF 
ACAT  I  PROGRAM  MANAGERS 


LCCM 

Models 

ACAT  1  Program  Managers 
with  over  11  years  of  experience 

No  Experience 
with  Model 

Thoughts  based  on 
Experience  with  Model 

Not  Familiar 
or  Not  Used 

Not  Useful  Useful 

One  of 
the  Best 

ACARA 

87% 

2% 

10% 

1% 

CASA 

78% 

2% 

18% 

2% 

EDCAS 

90% 

2% 

7% 

1% 

MAAP 

89% 

2% 

7% 

2% 

FLEX 

91% 

3% 

4% 

2% 

LCCA 

72% 

3% 

22% 

4% 

LCCH 

74% 

2% 

21% 

3% 

PRICE 

73% 

2% 

23% 

3% 

ZCORE 

92% 

2% 

3% 

0% 

ACEIT 

70% 

2% 

24% 

4% 

Note.  This  table  represents  an  opinion  survey  conducted  for  purposes  of  this  research: 
the  percentages  represent  input  from  887  survey  respondents,  543  of  whom  were 
current  or  former  DoD  PMs. 


use,  and  limitations  are  viewed  as  questionable  by  many,  including  DoD’s 
most  experienced  program  managers  (Table  1).  Confidence  in  these  models 
appears  to  have  waned. 

Sentiments  like  those  expressed  by  the  National  Aeronautics  and 
Space  Administration  (NASA,  2008)  are  common  among  many  acquisition 
professionals  with  comparable  years  of  experience  on  the  subject  of 
developing/relying  upon  the  accuracy  of  LCC  estimates  that  models  like 
these  provide. 

It  involves  using  incomplete,  inaccurate,  and  changing  data  for  an 
outmoded  &  ineffective  space  system  to  derive  the  precise  cost  of 
purchasing  an  unknown  quantity  of  an  undefined  new  space  system 
to  satisfy  an  overly  exaggerated  and  unvalidated  requirement 
at  some  time  in  the  future,  under  uncertain  conditions,  with  a 
minimum  of  funds,  (p.  17) 

Whatever  model  or  methodology  is  selected,  carefully  (and 
frequently)  applying  it  can  have  a  lasting  effect  on  cost  containment.  Of 
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primary  importance  is  the  selection  of  the  most  suitable  LCC  moclel(s). 
Each  characterizes  a  number  of  important  variables  a  little  differently. 
Nonetheless,  each  LCC  model  also  has  the  capacity  to  magnify  cost  drivers, 
early  and  often.  Regrettably,  Booz  Allen-Hamilton  reported  that  the  “real 
issue  is  one  of  obtaining  the  data  in  a  timely  manner  and  of  reducing  the 
redundant  data  collection  effort  needed  every  time  a  cost-effectiveness 
question  arises  in  the  decision-making  arena”  (Leggitt,  1981,  p.  13).  However, 
unless  PMs  alter  their  views  on  their  usefulness  and  frequency  of  use,  these 
models/methodologies  will  likely  have  less  influence  on  key  decisions. 

Fundamentally,  LCCM  is  actually  an  extraordinary  concept,  which  is 
generally  described  through  two  manifestations.  The  first,  LCC,  accounts 
for  research  and  development  costs,  investment  costs,  operating  and 
support  costs,  and  disposal  costs  over  the  system’s  entire  Life  Cycle.  The 
LCC  includes  not  only  the  direct  costs  of  the  acquisition  program,  but  also 
includes  indirect  costs  that  would  be  logically  attributed  to  the  program. 
The  second.  Total  Ownership  Cost  (TOC),  consists  of  LCC  elements  as 
well  as  other  infrastructure  or  business  process  costs  not  necessarily 
attributable  to  the  program  (DoD,  2008).  Understanding  all  the  costs  and 
all  the  implications  associated  with  LCCM  may  seem  intimidating.  So  many 
unknowns  and  so  many  combinations  and  permutations  come  into  play 
that  can  easily  vary,  making  it  difficult  to  quantify  any  system’s  total  costs, 
especially  when  it  matters  most— during  the  birth  of  a  program. 

In  2006,  to  raise  more  awareness,  DoD  elevated  the  ranking  of 
ownership  costs  to  a  Key  System  Attribute  (KSA)  in  anticipation  of  drawing 
more  attention  early  on  (Kobren,  2009).  Have  we  given  LCCM  enough 
attention  to  have  an  impact  though?  Probably  not.  And  if  not,  how  can  we 
garner  even  more  attention  and  emphasis  on  this  KSA?  Perhaps  we  should 
just  call  it  what  it  is— Aggregate  Management.  After  all,  it  aggregates 
everything  that  could  possibly  affect  the  cost  of  materializing  anything  that 
actually  gets  built  and  eventually  fielded  in  the  DoD. 

Investment  budgets  are  shrinking,  and  without  additional  attention, 
initial  concepts  designed  to  meet  some  requirement  might  take  a  lot 
longer  to  materialize  or  cost  a  whole  lot  more  to  produce  and  sustain— both 
problematic  scenarios  that  we  as  a  nation  can  ill  afford.  LCCM  needs  to  be 
somehow  re-energized.  Increasing  its  use  would  trigger  the  robust  part  of 
the  LCCM  challenge— encouraging  deeper  thinking,  acting  more  critically, 
and  pursuing  more  creative  methods  to  contain  overall  costs.  Years  earlier, 
Lt  Gen  James  T.  Stewart,  USAF  (Ret.),  indicated  one  of  the  threats  to  cost 
containment  and  described  it  as  “yo-yo  funding”  (Dapore  &  Bryant,  1984, 
p.  312)  that  persists  even  today  in  the  DoD’s  Planning,  Programming, 
Budgeting,  and  Execution  (PPBE)  process. 
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Exchange  with  Subject  Matter  Experts  (SMEs) 

The  authors  conducted  two  focus  sessions  with  a  handful  of  acquisition 
experts  who  teach  the  art  and  science  of  LCCM  and  cost  estimating.  Their 
experiences,  combined  with  frequent  contact  with  acquisition  colleagues 
inside  and  outside  the  classroom,  highlighted  specific  cost-containment 
issues  that  PMs  face  every  day. 

Their  first  meeting  was  with  the  Logistics  SMEs.  Each  SME  confirmed 
that  LCCM  issues  persist.  They  noted  LCCM  considerations  continue  to  be 
minimized  up  front  where  they  could  have  the  most  significant  impact. 
They  also  stressed  any  discussion  on  LCCM  tends  to  be  short-lived, 
especially  further  down  the  acquisition  continuum  and  after  initial  modeling 
(R. Burroughs,  personal  communication,  September  17,  2009). 

To  amplify  the  importance  of  LCCM,  the  SMEs  recommended  instituting 
an  LCC  breach  construct  (similar  to  the  intent  behind  Nunn-McCurdy 
breaches).  For  example,  if  a  program  exceeded  its  LCC  baseline  by  a  fixed 
cost  percentage  similar  to  the  construct  established  by  Nunn-McCurdy,  PMs 
would  have  to  report  any  infringement  to  Congress.  They  also  indicated 
it  would  be  beneficial  to  establish  a  formulary  similar  to  TRLs  where  a 
program  could  not  proceed  to  the  next  phase  until  it  demonstrated  some 
minimum  level  of  achievement  (M.  Sherman,  personal  communication, 
September  17,  2009).  Currently,  DoD  expects  LCC  reassessments  after  an 
initial  one  is  developed,  but  do  these  subsequent  updates  give  enough 
attention  to  cost  containment?  Not  explicitly. 

The  logistics  SMEs  emphasized  both  the  lack  in  LCCM  discipline  and 
the  absence  of  cross  communication  in  programs  that  generally  need  it 
the  most  throughout  a  program’s  life  cycle.  They  accentuated  that  funding 
allocations  and  key  decisions  typically  seem  to  be  focused  on  development 
and  not  sustainment.  And,  without  a  tool  to  respond  to  the  dynamic  nature 
of  LCC  that  accounts  for  all  costs,  including  Operations  and  Support  (O&S), 
there  will  be  little  forewarning  a  sustainment  breach  might  be  close  at  hand 
(M.  Sherman,  personal  communication,  September  17,  2009). 

A  widely  recognized  tenet  of  DoD  program  management  is  that  O&S 
costs  constitute  the  majority  of  a  program’s  total  costs— a  widely  recognized 
tenet  in  DoD  program  management.  As  recently  as  March  2007,  the  Cost 
Analysis  Improvement  Group  (CAIG)  reaffirmed  that  “projected  O&S  costs 
average  60-65  percent  of  projected  life-cycle  costs  after  reviewing  34 
Major  Defense  Acquisition  Programs,  or  MDAPs  (CAIG,  2007).  Just  as 
strikingly,  at  the  end  of  a  program’s  research  and  development  effort  and 
just  prior  to  production  or  operations,  95  percent  of  the  cumulative  LCC  has 
already  been  committed  (DOE,  1997).  So,  is  the  lack  of  attention  actually 
warranted  in  subsequent  life-cycle  phases  given  the  questionable  ability 
to  influence  O&S  costs?  The  authors  suspected  so,  but  were  anxious  to 
hear  and  consider  divergent  views  from  the  Budget,  Cost,  and  Financial 
Management  experts. 
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The  authors  next  met  with  four  Budget,  Cost  Estimating,  and  Financial 
Management  (BCEFM)  SMEs.  This  group  echoed  the  same  sentiment  voiced 
by  the  Logistics  SMEs:  Sustainment  tends  to  get  minimized  early  in  the 
development  phase.  Flowever,  they  added  that  the  "ilities”  are  generally  not 
well-defined.  They  stated  LCCM  typically  suffers  from  a  lack  of  sufficient 
cost  detail  to  adequately  address  sustainment  costs  that  predominate  once 
systems  find  their  way  into  operations  (R.  Morig,  personal  communication, 
September  22,  2009). 

The  BCEFM  SMEs  quickly  reached  a  consensus  on  one  of  the  major 
obstacles  to  cost  containment.  They  stated  funding  instability  makes  cost 
containment  an  insurmountable  prospect.  Already  faced  with  many  other 
daily  programmatic  challenges,  they  asserted  that  funding  instability, 
typically  manifested  by  perpetual  budget  cuts,  creates  a  gyrating  funding 
baseline  on  top  of  other  strategic  concerns  including: 

•  Industry  partners  who  are  not  necessarily  motivated  by  cost 
containment 

•  Frequent  changes  in  requirements 

•  Internal  staffing  shortfalls  that  are  sometimes  tough  to  fill 

•  Lack  of  certain  key  functional  experience  in  program  offices 

•  Cultural  realities  that  emphasize  program  survival  over 
program  affordability. 

The  BCEFM  SMEs  also  affirmed  if  PMs  found  a  cost  metric  that  had  a  strong 
influence  in  controlling  costs  well  after  the  “truthful  predictions,”  it  would 
be  widely  used  and  could  perhaps  help  contain  costs  (J.  Rego,  personal 
communication,  September  22,  2009).  EVM  satisfies  the  forecasting 
piece  of  the  equation,  but  without  specific  and  practical  motivational 
methods  that  help  contain  costs,  its  usefulness  is  questionable.  So,  do 
those  specific  methods  exist  today?  The  answer  is  yes.  Contract  incentive 
strategies  are  one  of  many  tools  available,  and  have  been  used  extensively 
in  DoD  to  help  curb  some  of  the  escalating  technical  risks  and  associated 
costs.  However,  they  have  tended  to  provide  more  short-term  gains 
than  the  ones  needed  for  longer  term,  and  more  enduring  outcomes 
in  the  past  few  years,  especially  when  technology  maturity  is  so  fluid 
(GAO,  2005). 


LCC  IN  PRACTICE  TODAY 

Today,  in  the  context  of  containing  costs  in  acquisition  programs  in 
the  DoD,  PMs  are  compelled  to  address  LCCM  across  their  program’s 
life  cycle.  As  mentioned  earlier  though,  well  before  a  PM’s  arrival  much 
of  the  projected  life-cycle  costs  for  future  systems  or  products  is  rooted 
in  decisions  made  during  the  early  phases  of  advanced  planning  and 
conceptual  design  (Blanchard,  1992).  Consequently,  initial  LCC  assessments 
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have  always  been  a  key  component  of  a  program’s  “go/no  go”  decision 
process  since  they  address  a  program’s  affordability,  and  are  ultimately 
dependent  on  the  military  department’s  (or  agency’s)  ability  to  secure  the 
necessary  funding.  Each  military  department  and  agency  gives  LCCM  a  lot 
of  attention  at  the  beginning  of  a  system’s  life  cycle.  However,  in  addition  to 
LCCM  concerns,  military  departments  and  agencies  must  balance  today’s 
operational  needs  with  future  requirements,  and  not  neglect  more  capable 
systems  still  in  various  stages  of  development— designed  to  either  boost 
current  system  performance  or  meet  new  warfighter/user  requirements. 

LCC  projections  are  not  expected  to  be  dormant  once  PMs  take 
charge.  Title  10  of  the  United  States  Code  §  2434  requires  the  Secretary 
of  Defense  to  consider  an  independent  Life  Cycle  Cost  Estimate  (LCCE) 
before  approving  Engineering  and  Manufacturing  Development  (EMD), 
or  Production  and  Deployment  (P&D)  of  an  MDAP.  In  practice,  LCC  gets 
looked  at  closely  via  an  assortment  of  predictive  analyses  (probabilistic 
and  deterministic)  that  sometimes  can  be  difficult  to  absorb.  So  much  so, 
that  it  is  generally  left  to  the  experts  to  decipher.  Very  few  PMs  ever  find 
themselves  digging  into  LCC  parameters.  Besides,  they  have  the  experts 
in  their  respective  program  offices  who  analyze  and  weigh  the  output. 
Even  so,  many  variables  make  it  sometimes  difficult  for  even  the  experts 
to  fully  quantify.  The  experts,  who  generally  populate  the  models  with 
key  assumptions,  do  their  best  to  leverage  the  behavior  of  analogous 
systems.  Still,  quantifying  all  the  assumptions  is  a  daunting  task  when  so 
many  parameters  are  so  variable  or  have  not  been  captured  or  qualified. 
Ultimately,  the  responsibility  resides  with  the  PM  to  embrace  LCC  estimates, 
but  do  they  and  their  staffs  revalidate  these  estimates  on  a  more  routine 
basis?  Do  they  dive  deeper  into  the  basis  of  the  original  LCC  estimate  and 
make  any  necessary  adjustment(s)  to  contain  costs? 

PMs  recognize  that  LCC  generally  starts  out  with  an  “inferred”  cost- 
containment  element  before  their  programs  leave  the  initial  approval 
process  gate.  What  happens  later  is  a  combination  of  art  and  science  mixed 
with  some  uneasiness.  PMs  are  expected  to  quantify  the  anticipated  costs  of 
their  development  system  across  the  Future  Years  Defense  Program  (FYDP). 
For  ACAT  1C  and  ID  programs,  LCC  is  carefully  revisited  by  Congress  in  the 
context  of  Program  Acquisition  Unit  Cost  (PAUC)  when  costs  escalate  by 
at  least  15  percent  or  more  of  the  current  baseline,  or  30  percent  or  more 
over  the  original  baseline  (DAU,  2009a,  p.  31).  While  operational  costs  can 
be  extrapolated  as  a  derivative,  they  seem  to  be  temporarily  suspended 
from  the  equation  since  operational  costs  cannot  yet  be  easily  reconciled. 
What  happens  further  down  the  acquisition  trail  for  most  programs  does 
not  necessarily  force  PMs  to  either  necessarily  challenge  the  assumptions 
that  were  part  of  the  original  LCC  equation  or  consider  that  LCC  is  such  a 
dynamic  process. 

After  Milestone  B  (formal  initiation  of  an  acquisition  program),  PMs 
tend  to  narrow  their  focus  on  managing  their  programs  day-to-day.  This 
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day-to-day  strategy  is  about  program  survival.  PMs  dwell  on  cost,  schedule, 
and  performance  parameters  in  the  face  of  too  little  funding,  too  little 
schedule  flexibility,  and  too  many  technology  hurdles.  If  LCC  models 
are  seen  as  an  initial  forecasting  apparatus  only  to  give  a  reasonable 
grounding  of  all  known  costs— but  not  necessarily  designed  to  contain 
costs— how  could  cost,  schedule,  and  performance  become  more  tightly 
integrated  into  the  overall  LCCM  equation?  And,  what  about  CAIV?  Where 
does  it  fit  in?  As  originally  envisioned,  CAIV  was  designed  to  give  PMs 
the  flexibility  to  balance  all  the  factors  that  could  help  contain  costs— but 
has  it?  What  do  PMs  have  to  say  about  CAIV?  How  are  LCC  and  CAIV 
related?  Are  they  related?  What  do  PMs  think  about  these  questions?  Their 
perspectives  follow. 

Survey  Findings 

The  objective  data  generated  by  this  opinion  survey  confirmed  what 
some  earlier  studies  found  in  LCCM.  In  addition,  the  data  offered  quite  a 
few  other  interesting  perspectives  as  well,  especially  in  the  way  PMs  view 
LCCM  and  CAIV  regarding  cost-containment  principles.  The  survey  also 
reinforced  how  PMs  unevenly  apply  LCCM  principles  and  cost-containment 
strategies  across  their  programs. 

Even  though  the  opinions  expressed  in  this  survey  were  based  on 
fundamental  beliefs,  opinions  invariably  drive  decisions  since  they  are 
inextricably  linked  to  “experiences”— an  imperative  in  the  DoD’s  acquisition 
enterprise,  and  one  of  the  key  factors  designed  to  help  meet  the  certification 
requirements  of  the  acquisition  corps.  In  other  words,  opinions  matter  in 
the  acquisition  profession  when  such  opinions  are  steeped  in  years  of 
acquisition  experience.  Burrowing  into  the  invaluable  experiences  that 
have  shaped  DoD’s  current  PM  workforce  can  also  be  a  very  meaningful 
bellwether.  In  this  particular  survey,  PMs  provided  specific  narrative 
comments  that  acknowledged  certain  cost-containment  hurdles.  The  survey 
also  found  a  couple  of  misconceptions  regarding  the  use  and  usefulness  of 
some  of  these  cost-containment  tools  in  the  Tool  Kit.  The  discussion  that 
follows  addresses  noteworthy  findings. 


LCC  MODEL  FAMILIARITY  AND  EXPERIENCE 

When  887  PMs  were  asked  to  rate  the  LCC  models  that  they  had 
previously  used,  many  were  simply  unfamiliar  with  the  models.  Provided 
below  are  representative  comments  from  the  opinion  survey  results 
(Table  1). 

Sorry,  just  not  that  familiar  with  the  models.  Somebody  else  uses 
them  and  provides  data  to  me. 
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As  a  PM,  I  have  not  been  involved  vyith  the  detailed  execution  of  the 
specific  model  used  to  derive  cost  estimates.  In  many  instances, 
costs  and  cost  estimates  vyere  derived  from  legacy  numbers  of  the 
previous  program. 

To  be  honest,  not  my  field  of  expertise,  and  I  am  only  familiar  with 
the  tools  to  the  extent  my  team  uses  them. 

I  have  no  first-hand  knowledge  of  any  of  these  systems/models. 


USEFULNESS  OF  LCC  MODELS 

PMs  believed  that  the  P&D  and  O&S  phases  are  better  predictors 
of  costs,  while  the  Technology  Development  (TD)  and  EMD  phases  are 
generally  the  most  influential  in  driving  decisions.  Contrary  to  what  the 
DoD  would  prefer,  they  did  not  believe  the  pre-acquisition  phases  (Materiel 


TABLE  2.  LIFE  CYCLE  PHASES  WHERE  LIFE  CYCLE  COST  MODELS 
MADE  AN  IMPACT:  VIEWS  OF  ACAT  I  PROGRAM  MANAGERS 


In  what  Life  Cycle  Phases  are  the  Life  Cycle  Models ... 


A 

A 

L  A 

IOC 

FOC 

Materiei  Solution 

Technology 

Engineering  and 

Production  and 

Operations  and 

mj  Analysis 

Development 

Manufacturing  Development 

Deployment 

Support 

^  Materiel  Dev.  Decision 

AoA 

^  Post  PDR  Assess.  ^  Post  CDR  Assess. 

PDR  PDR  CDR 

^  FRP  Decision  Review 

Pre-Systems  Acquisition  Systems  Acquisition  Sustainment 


MSA 

TD 

EMD 

P&D 

O&S 

None 

DK 

a  good  cost 

14 

19 

59 

68 

67 

45 

31 

predictor? 

most  influential  in 

45 

66 

91 

41 

34 

20 

28 

driving  decisions? 

suitable  for  cost 

15 

28 

52 

64 

48 

56 

33 

containment? 

significantly 

65 

82 

107 

58 

60 

18 

32 

underestimated? 


Note.  MSA=Materiet  Solution  Analysis:  TD=Technology  Development;  EMD=Englneering  and 
Manufacturing  Development;  P&D=Production  and  Deployment;  OSS=Operations  and  Support; 
None=No  Model  Is  a  Good  Predictor;  DK=  Don't  Know  Which  Mode!  Is  a  Good  Predictor. 


2  5  6 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


Solution  Analysis  and  TD)  are  suitable  for  cost  containment  given  their 
inability  to  qualify  let  alone  quantify  some  of  the  major  "unknowns.”  More 
importantly,  by  the  time  their  programs  entered  EMD,  a  large  number  of 
PMs  declared  that  LCC  models  have  significantly  underestimated  costs. 
PMs  also  stated  these  models  need  more  precision  in  the  early  stages  of 
program  initiation  since  they  drive  so  many  future  decisions  (Table  2). 
Organizations  like  the  CAIG  recommended  that  PMs  should  seek  more 
research  that  focused  on  “scrubbing  development  and  procurement, 
more  detailed  analysis  of  sustainment  profiles,  and  identification  of  causal 
H  factors”  (CAIG,  2007). 

Representative  Narrative  Comments.  A  sampling  of  comments  on  the  way  PMs 
view  LCCM  and  its  cost-containment  principles  follows. 

Most  models  have  many  assumptions,  and  those  assumptions  are 
not  monitored  over  time;  and  risks  are  not  addressed  to  keep  the 
assumptions  valid,  so  the  models  are  not  valuable  when  decision 
makers  really  need  the  information. 

LCC  for  O&S  appears  to  be  generally  unrealistic. 

As  programs  proceed  along  their  life  cycle,  LCC  doesn’t  seem  to 
be  appropriately  updated. 

LCCM  never  captures  changes  allowed/forced  on  programs,  and 
fails  to  "predict"  well.  Models  are  used  early  on,  but  eventually 
lose  influence  as  "inertia"  takes  over  and  programs  enter  "make 
the  best  of  it  mode." 

Overly  optimistic  estimates. 

||  I  No  one  seems  to  put  the  thought  and  time  into  a  thorough  estimate 

I  of  determining  LCC. 

No  one  seems  to  update  LCC  and  use  it  as  a  yardstick. 


MAJOR  OBSTACLES  TO  COST  CONTAINMENT 

Of  the  many  typical  challenges  that  PMs  face,  five  obstacles  accounted 
for  a  noticeable  majority  of  the  reasons  that  made  cost  containment  difficult 
to  overcome.  Those  five  standing  in  the  way  included  requirements  creep, 
underfunded  programs,  annual  budget  fluctuations,  ambitious  program 
schedules,  and  too  many  policy  and  bureaucratic  obstacies  (Figure  7). 
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FIGURE  7.  PROGRAMMATIC  OBSTACLES  THAT  MAKE  COST 
CONTAINMENT  DIFFICULT  TO  OVERCOME:  VIEWS  OF  ACAT  I 
PROGRAM  MANAGERS 


Absence  or  lack  of  useful 
cost  containment  model 

Lack  of  certain  functionai 
expertise  and  experience 
in  program  offices 


Reduced  procurement  quantities 


Requirements  creep 


Extended  procurements 

Excessive  certification 
requirements 


Overiy  compiex  software 


Poiicy  and  bureaucratic  obstacies 


Ambitious  program  scheduies 


REVISIT  RATES  FOR  LCC  ESTIMATES 

Despite  whether  revisiting  LCC  estimates  was  viewed  as  a  burden  or 
resource  constraint,  about  half  of  the  PMs  routinely  or  frequently  reviewed 
their  program’s  LCCs  unless  in  preparation  for  an  upcoming  milestone 
review  (Figure  8).  While  a  great  forcing  function,  performing  LCC  updates 
only  in  preparation  for  the  next  milestone  is  probably  too  late  to  significantly 
influence  cost  containment.  However,  PEOs  and/or  senior  managers 
showed  even  less  interest  in  LCC  estimates  other  than  preparation  for  the 
next  milestone  (Figure  8).  Without  more  frequent  and  intensive  reviews  by 
either  PMs  or  PEOs,  the  ability  to  make  cost  adjustments  becomes  more 
difficult  to  defend. 

Representative  Narrative  Comments.  A  sampling  of  comments  on  revisiting  LCC 
highlights  this  seemingly  low  level  of  interest  in  LCC  estimates  other  than 
for  milestone  reviews. 
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FIGURE  8.  REVISITING  LIFE  CYCLE  COST  ESTIMATES:  VIEWS  OF 
ACAT I  PROGRAM  MANAGERS 


Don't  know 


Never 


PM  LCC  Review 


Don't  know 


PEO  LCCC  Review 


The  costs  that  are  of  the  most  concern  to  me  are  those  in  the 
immediate  execution  year.  I  have  considered  out-year  costs  but 
not  as  much  as  I  should  have. 

My  focus  is  on  providing  most  capability  within  budget,  not  on 
future  life-cycle  costs. 

General  knowledge  on  cost  containment  among  all  program  office 
personnel  is  very  low. 

Many  of  the  cost  growths  are  based  on  not  really  understanding 
the  requirements  and  instead  based  on  assumptions  on  both  sides. 
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FIGURE  9.  HOW  COST  DRIVERS  RANK  IN  ORDER  OF 
SIGNIFICANCE:  VIEWS  OF  ACAT  I  PROGRAM  MANAGERS 


a.  Change  in  requirements 

b.  Immature  technology 

c.  Overly  ambitious  schedule 

d.  Funding  instabilty 

e.  Artificially  low  cost  estimates 

f.  Reduction  in  procured  units 

g.  Workforce  experience 

h.  Lack  of  enough  testing 


SIGNIFICANT  COST  DRIVERS 

Identifying  and  knowing  the  significance  of  key  cost  drivers  are 
paramount.  Otherwise,  the  ability  to  contain  costs  could  easily  weaken. 
When  asked  how  they  would  rate  the  significance  of  many  of  the  classic 
cost  drivers,  PMs  expressed  that  immature  technology,  funding  instability, 
changing  requirements,  artificially  low  cost  estimates,  and  overly  ambitious 
schedules  were  the  most  significant  (Figure  9).  With  the  addition  of 
artificially  low  cost  estimates  and  too  many  policy  and  bureaucratic 
obstacles,  these  were  the  same  obstacles  that  made  cost  containment 
difficult  to  overcome  when  an  even  wider  selection  of  survey  choices  was 
posed  to  PMs  in  an  earlier  question  (Figure  1). 


CONNECTION  BETWEEN  CAIVAND  LCC 

CAIV  is  another  key  tool  available  to  help  contain  costs  as  previously 
discussed.  It  gives  PMs  a  flexible  instrument  to  help  quantify  the  undeniable 
relationship(s)  between  certain  performance  requirements  and  realistic  cost 
constraints.  Flowever,  only  65  percent  of  the  PMs  acknowledged  either  a 
“strong”  or  "moderate”  connection  to  LCC  (Figure  10).  Subsequently,  PMs 
might  see  CAIV  as  a  quick  fix  only  and  not  fully  appreciate  the  extent  of  the 
long-term  gain;  not  believe  there  is  a  long-term  gain;  or  perhaps  not  fully 
believe  in  the  concept  as  a  whole. 


Representative  Narrative  Comments.  A  sampling  of  comments  on  the  relationship 
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FIGURE  10.  STRENGTH  OF  CONNECTION  BETWEEN  CAIV  AND 
LCC;  VIEWS  OF  ACAT  I  PROGRAM  MANAGERS 


None  Don't  know 


Moderate 


between  CAIV  and  LCC  shows  a  program  management  community  less 
comfortable  with  CAIV  as  a  cost  control  tool. 

Strong  in  theory  but  weak  in  practice. 

I  think  the  relationship  between  LCC  and  CAIV  has  been  diminished. 

I’ve  never  seen  CAIV  used  to  contain  costs  on  a  program. 

I  don’t  believe  CAIV  has  anything  to  do  with  CAIV.  It’s  an  artificial 
constraint  that  prevents  the  PM  from  meeting  the  requirements. 

I  didn’t  see  CAIV  used  in  any  organized  way  because  hardly  anyone 
on  the  PM  team  has  enough  practical  experience. 

Unfortunately,  the  CAIV  tool  of  last  resort  became  common  to 
overcome  cost  overruns  due  to  funding  stability  and  poor  execution. 

CAIV  trades  are  rarely  supported  by  the  requirements  community. 
The  requirements  community  is  99  percent  focused  on  capability 
and  mildly  interested  in  long-term  O&S  cost-reduction  efforts. 


TRAINING  CHALLENGES 

PMs  stated  a  need  for  additional  training,  primarily  LCCM  and  Risk 
Management  training,  to  help  them  better  contain  costs  (Figure  11). 
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FIGURE  11.  ADDITIONAL  TRAINING  NEEDED  TO  BETTER 
CONTAIN  COSTS:  VIEWS  OF  ACAT  I  PROGRAM  MANAGERS 
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Recommendations 

To  reconcile  some  of  the  shortcomings  of  LCC  and,  just  as  importantly, 
better  prepare  PMs  to  contain  costs  and  achieve  more  successful  acquisition 
outcomes,  the  authors  of  this  research  recommend  the  following: 

•  Take  the  chill  out  of  cost  containment  and  re-energize 
LCCM.  Make  it  everyone’s  business.  Even  though  PMs 
cannot  serve  as  LCC  experts,  they  and  their  teammates 
should  know  the  basis  of  their  own  LCC  estimates 
throughout  their  program’s  life  cycle,  and  not  wait  until  the 
next  milestone  to  make  any  necessary  adjustment(s). 

•  Elevate  LCC  to  a  KPP  (Key  Performance  Parameter)— it 
will  compel  more  PMs  and  senior  personnel  to  rigorously 
exercise  LCCM  principles.  Establishing  LCC  as  a  KSA  is 
not  enough. 

•  Continuously  challenge  assumptions. 

•  Base  cost  decisions  on  programmatic  realities  and  more 
current  data  since  these  influence  LCC  outcomes. 

•  Establish  an  LCC  Continuous  Learning  Model  (CLM)  that 
amplifies  the  objectives  and  characteristics  of  an  LCC 
model,  and  identifies  the  family  of  LCC  models  that  best 
apply  where,  how,  and  when. 
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•  Add  an  LCC  best  practice  link  to  each  functional 
Community  of  Practice  (CoP)  where  PMs  can  learn 
from  others. 

•  Establish  LCCM  trip  wires  throughout  a  program’s  life 
cycle,  and  do  not  penalize  PMs  for  reporting  unfavorable 
but  essentially  accurate  program  information  to  seniors  or 
higher  headquarters. 

•  Reward  and  incentivize  PMs  for  containing  and/or 
lowering  costs. 

•  Develop  cost-containment  strategies  that  are  carefully 
evaluated  and  painless  to  execute. 

•  Embrace  innovation  and  dismiss  mundane  strategies  that 
guarantee  less-than-optimal  outcomes. 

•  Promote  more  CAIV.  Conceptually,  CAIV  was  placed  into 
the  acquisition  arsenal  to  give  PMs  a  little  more  latitude 
with  performance  versus  cost  trade-offs.  As  ADM  Mike 
Mullen,  USN,  Chairman  of  the  Joint  Chiefs  of  Staff,  recently 
said  at  the  Program  Executive  Officer/Systems  Command 
Commanders’  Conference  at  Fort  Belvoir,  Virginia,  on 
November  4,  2009,  “The  acquisition  community  and 

the  warfighter  will  have  to  jointly  accept  the  80  percent 
solution. ..we  have  to  be  realistic  with  what  we  can  afford.” 

(Mullen,  2009). 

•  Let  PMs  lead.  PMs  have  the  knowledge,  skill,  and  ability  to 
carefully  guide  their  programs  in  the  face  of  a  complex  and 
difficult  environment. 

Conclusions 

This  research  reinforced  the  many  contrasting  perspectives  that  PMs 
possess  with  respect  to  cost  containment  and  their  ability  to  influence 
and/or  control  it.  As  originally  conceived,  understanding  the  usefulness 
and  criticality  of  LCCM  can  have  a  major  impact  on  weapons  systems 
developments  by  keeping  a  lid  on  rising  costs— a  growing  necessity.  The 
acquisition  environment  will  invariably  change.  Budgets  will  shrink;  fewer 
new  systems  will  be  built  and  fielded;  more  pressure  will  be  exerted  on 
extending  and  sustaining  current  systems;  and  more  pressure  can  be 
expected  on  containing  costs— much  more  pressure.  The  remaining  weapons 
systems  under  development  will  come  under  political  fire.  As  external 
scrutiny  swells,  programmatic  decisions  will  be  challenged  since  there 
will  be  so  much  more  information  immediately  available  about  emerging 
systems.  So,  how  can  PMs  once  and  for  all  silence  the  skeptics  and  achieve 
positive  acquisition  outcomes?  For  starters,  they  can  shock  the  critics  by 
challenging  the  programmatic  “cost  status  quo”  at  every  juncture  and  not 
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just  the  major  milestones.  They  can  no  longer  “kid  themselves”  about  vyhat 
something  is  going  to  cost,  as  Under  Secretary  of  Defense  for  Acquisition, 
Technology  and  Logistics  Ashton  Carter  recently  stated  (Carter,  2009). 
They  can  increase  programmatic  “cost  accuracy”  by  better  understanding 
and  re-energizing  one  key  cost-containment  practice  that  has  seen  less 
action  or  become  ineffective  in  recent  years— LCCM.  Inarguably,  yo-yo 
funding  will  continue.  Poor  outcomes  need  not.  DoD  cannot  afford  more 
of  the  same.  Changes  to  DoD  5000.02  that  now  call  for  Preliminary  Design 
Reviews  (PDR)  prior  to  Milestone  B,  and  earlier  measured  prototyping  to 
lower  out-year  costs  will  go  a  long  way.  Warfighters  need  every  penny 
applied  to  capability,  not  cost  overruns.  Ultimately,  PMs  and  their  staffs 
must  be  more  introspective  and  tightly  integrate  the  art  and  the  science  of 
containing  costs  in  the  face  of  global  economic  changes.  It’s  time  to  take 
the  chill  out  of  containing  costs.  DoD  depends  on  it;  our  nation  depends  on 
it;  and  the  warfighters  need  to  count  on  it. 
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APPENDIX 

List  of  Abbreviations  and  Acronyms 

AoA  Analysis  of  Alternatives 

ACAT  Acquisition  Category 

ACWP  Actual  Cost  of  Work  Performed 

ADM  Admiral 

BAC  Budget  at  Completion 

BCEFM  Business,  Cost  Estimating,  and  Financial  Management 

BCWP  Budget  Cost  for  Work  Performed 

BCWS  Budget  Cost  for  Work  Scheduled 

CAIG  Cost  Analysis  Improvement  Group 

CAIV  Cost  as  an  Independent  Variable 

CARD  Cost  Analysis  Requirements  Description 

CDR  Critical  Design  Review 

EAC  Estimate  at  Completion 

DAU  Defense  Acquisition  University 

CDD  Capability  Development  Document 

CPD  Capability  Production  Document 

DoD  Department  of  Defense 

DOE  Department  of  Energy 

EMD  Engineering  and  Manufacturing  Development 

EVM  Earned  Value  Management 

FOC  Full  Operational  Capability 

FRP  Full  Rate  Production 

FYDP  Future  Years  Defense  Program 

GAO  Government  Accountability  Office 

ICD  Initial  Capabilities  Document 

IOC  Initial  Operational  Capability 

KPP  Key  Performance  Parameter 

KSA  Key  System  Attribute 

LCC  Life  Cycle  Cost 

LCCE  Life  Cycle  Cost  Estimate 

LCCM  Life  Cycle  Cost  Management 
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A  NEW  ALPHA-OMEGA 
MAP  FOR  ACQUISITION 
TEST  AND  EVALUATION 

f  George  Axiotis 

Department  of  Defense  Acquisition  Test  and  Evaiuation  (T&E) 
has  remained  the  gatekeeper  to  Major  Defense  Acquisition 
Program  production  since  its  formaiization  over  25  years 
ago.  Under  T&E’s  oversight,  the  types,  methods,  and  sources 
for  warfighting  systems  have  significantiy  evoived  to  meet/ 
counter  the  nation’s  security  chaiienges.  The  DoD  has  studied 
and  recommended  actions  to  acceierate  Acquisition  Reform 
for  decades,  whiie  oniy  “tweaking  at  the  margins”  for  T&E. 

Now  is  the  time  for  DoD  to  consider  a  new  approach  to  T&E, 
steering  away  from  the  “buy”  decision  to  the  more  reievant 
“acceptance”  and  “operationai”  domains.  This  articie  outiines 
the  issues  and  proposes  a  new  “Aipha-Omega”  map  for  T&E 
that  charts  the  way  ahead  for  how  DoD  actuaiiy  procures  its 
weapon  systems. 


Keywords:  Test  &  Evaluation,  Development  Testing, 
Operational  Testing,  Acquisition  Reform, 
Alpha-Omega  Mode! 
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Department  of  Defense  (DoD)  leaders  and  numerous  reform  studies 
criticize  the  department’s  acquisition  process  for  its  inability  to  control 
spiraling  costs  and  delays  in  getting  systems  to  the  user.  In  response, 
Acquisition  Reform  efforts  to  date  focus  on  better  requirements,  efficient 
resources  planning,  as  well  as  increasing  feedback  and  accountability  (DoD, 
2004a,  pp.  2-7).  DoD  Acquisition  Test  and  Evaluation  (T&E),  comprising  the 
formal  processes,  policies,  personnel,  equipment,  facilities,  and  consumables 
necessary  to  develop,  certify,  test,  and  evaluate  defense  systems  for 
production,  has  faced  much  of  this  criticism.  In  response,  T&E  reform  focuses 
on  process  streamlining,  reducing  overhead,  and  further  integration  within 
the  engineering  process  to  better  support  the  “buy”  decision. 


Background 

The  defense  acquisition  process,  using  T&E  as  a  gatekeeper,  is  relatively 
unchanged  since  the  Packard  Commission  recommendations  (DoD,  1986a; 
DoD,  1986b,  p.  11).  Yet,  the  methods  and  players  of  DoD  acquisition  have 
fundamentally  changed.  Just  as  the  Pentagon  is  embracing  a  new  map  for 
the  application  of  military  power  based  on  an  active  strategy  for  the  world 
as  it  is  today,  DoD  needs  a  similar  active  strategy  for  determining  who  is 
best  served  by  T&E  (Barnett,  2004). 

This  article  examines  the  fundamental  influences  on  Acquisition  T&E, 
the  results  of  major  studies  to  date  on  reforming  Acquisition  T&E,  and 
concludes  that  the  current  emphasis  on  “buying”  as  the  raison  d'etres  for 
T&E  must  be  replaced  with  a  new  two-tiered  framework  and  leadership  that 
better  supports  both  contractual  necessity  and  operations  in  the  field.  A 
new  world  of  defense  systems  acquisition  is  emerging,  driving  the  need  for 
a  new  map  for  T&E  based  on  acceptance  and  operation,  which  I  suggest  is 
the  Alpha  and  Omega  of  a  new  T&E  order. 


GOOD  INTENTIONS 

Acquisition  T&E  follows  a  linear  engineering  approach  to  reduce 
risk,  building  insight  to  meet  contract  delivery,  assessing  the  delivered 
configuration  in  operational  environments,  verifying  that  the  system  works, 
and  certifying  that  the  system  warrants  further  procurement  (DoD,  2008a, 
pt.  1,  p.  12,  &  end  6,  pp.  51-53).  Acquisition  T&E  is  Service-  and  system¬ 
centric,  managed  through  dedicated  funding  and  contract  vehicles  with 
both  Development  Testing  (DT)  and  Operational  Testing  (OT)  supporting 
the  “buy”  decision.  Public  Law  delays  the  decision  beyond  initial  production 
until  a  submission  of  a  report  following  OT: 
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The  Secretary  of  Defense  shall  provide  that  a  major  defense 
acquisition  program  may  not  proceed  beyond  low-rate  initial 
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production  until  initial  operational  test  and  evaluation  of  the 
program  is  completed.  (Operational,  2009). 

Yet  for  all  its  formality,  T&E  plays  a  weak  role  in  controlling  what 
the  department  actually  buys.  The  current  DoD  acquisition  policy  allows 
initiation  of  low-rate  initial  production  (LRIP)  just  after  Critical  Design 
Review.  What  test  results  are  available  supports  this  decision,  but 
completion  of  testing  or  successful  results  are  not  formal  prerequisites. 

Note  that  DoD  has  no  "Approval  for  Service  Use”  decision  based  on 
successful  test  results.  The  only  legal  hurdles  to  proceeding  beyond  LRIP  are 
to  1)  submit  a  report  to  Congress  based  on  an  OT,  and  2)  seek  approval  of 
the  test  plan  in  advance  from  the  Director,  Operational  Test  and  Evaluation 
(DOT&E)  (Operational,  2009).  Again,  as  with  LRIP,  there  is  no  requirement 
to  pass  the  test,  only  to  conduct  it.  Acquisition  T&E  today  operates  within  a 
“buy”  construct  that  neither  demands  minimal  user-oriented  testing  before 
beginning  the  production  process,  nor  formalizes  the  full  commitment  to 
production  through  successful  completion  of  testing. 


EMPHASIS  ON  SPEED 

DoD  Acquisition  Reform  since  the  mid-1990s  emphasizes  faster  cycle 
times  through  efficient  management,  capitalizing  on  emergent  technologies, 
fielding  of  early  capabilities,  and  continuous  product  evolutionary  cycles. 
The  objective  is  to  maintain  competitive  edge  by  getting  to  the  "buy” 
decision  faster.  Yet,  in  World  War  II  compelling  need  put  emphasis  on 
production,  with  T&E  supporting  it.  Production  changes,  additional 
requirements,  and  performance  shortfalls  based  on  experience  in  the  field 
were  the  foundations  for  block  upgrades  (Gropman,  1997,  pp.  11,  41,  44, 
100, 115).  Over  60  years  later,  the  Mine  Resistant  Ambush  Protected  vehicle 
procurement  mimics  this  approach  with  an  emphasis  on  T&E  supporting 
production  (GAO,  2008,  pp.  8-9). 

DoD  works  to  balance  procurement  for  both  a  longer  term,  near-peer 
threat,  as  well  as  near-continuous  engagement  against  a  less  defined 
extremist  threat  (Chao,  2009).  Senior  leaders  struggle  with  the  current 
acquisition  construct,  and  if  given  the  choice  would  favor  one  that  responds 
better  to  ill-defined  threats,  requirements  that  grow  and  change  rapidly, 
and  technologies  that  evolve  many  times  within  the  development  cycle 
(Testimony,  2009).  Today,  as  in  WWII,  the  focus  is  getting  the  right 
capability  to  the  field  faster,  but  speed-to-user  is  not  enough  to  drive 
significant  change  in  the  current  T&E  process. 


A  NEW  ENVIRONMENT-REALLY 

DoD  systems  development  and  acquisition  have  undergone  profound 
change  in  recent  years,  brought  about  by  industry  practice  and  government 
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policy  that  sets  the  new  environment  for  T&E  (Defense  Science  Board, 
2008,  pp.  6, 16).  The  following  discussion  provides  an  illustrative  snapshot 
of  the  systems  development  and  acquisition  process: 


REQUIREMENTS  PROCESS 

Requirements  have  steered  away  from  the  primacy  of  technocrats, 
planners,  and  buyers  to  the  current  end-user  (DoD,  2004a,  pp.  2-7). 
Component  Commanders  present  unique  challenges  as  they  focus  on 
near-term  needs,  have  different  visions  for  how  requirements  are  met,  and 
have  diverse  views  on  how  advanced  technology  can  be  used.  These  users 
are  less  concerned  with  technology  nuances,  industrial  influences,  and 
specific  capabilities.  Yet,  to  manage  successful  acquisition,  specificity  is 
critical  for  configuration  design  and  engineering. 


SYSTEMS  DEVELOPMENT 

Systems  are  more  complex,  and  the  ability  to  characterize  fully 
end-state  performance  before  fielding  is  a  challenge.  Lead  Systems 
Integrators  (LSI)  have  increasingly  less  insight  into  the  subsystems 
they  are  integrating  and  thus  less  confidence  in  understanding,  and 
certifying  to,  actual  systems-of-systems  performance.  The  burden 
increasingly  falls  back  to  the  department  for  resolution,  accompanied 
by  risks  not  just  from  increasingly  complex  systems-of-systems 
integration  with  differing  maturity,  but  also  the  globalization  of  defense 
industrial  capability. 


INDUSTRIAL  BASE 

The  consolidation  of  the  Defense  Industrial  Base  through  the  1990s  has 
left  DoD  with  fewer  options  for  competitive  development  of  major  capital 
systems.  This  drives  systems  to  take  on  inherent  design,  engineering, 
production,  and  management  practices,  with  less  government  insight.  Key 
components  and  materials  will  increasingly  be  available  only  from  foreign 
sources,  inevitably  leading  to  less  control  of  the  design  and  engineering 
(Guay,  2007,  pp.  66-67).  The  emphasis  shifts  from  pre-production  to  as- 
delivered  product  adequacy. 


MISMATCHED  ACQUISITION  STRATEGIES 

The  Department  of  Defense  Instruction  (DoDI)  5000.02  relies  on 
technologies  being  wrung  out  before  initiating  development,  competitive 
prototyping  used  to  find  the  "best  of  breed,”  and  due  diligence  through  T&E 
before  production.  Yet,  technology  evolves  too  quickly  to  tie  acquisitions 
to  fixed  baselines,  with  initial  units  differing  in  performance  and  utility 
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from  those  later  on  in  production.  LSIs  today  deliver  systems  comprised 
of  subsystems  in  various  levels  of  maturity  and  product  life  cycles. 


NON-SERVICE-UNIQUE  SYSTEMS 

In  spite  of  the  rhetoric,  DoD  does  not  buy  capability— it  buys  “things” 
(systems)  that  are  married  to  others,  and  along  with  end  users  (warfighters) 
form  warfighting  capabilities.  While  DoD  is  procuring  more  “joint”  systems 
than  ever  before,  the  vast  majority  are  still  Service-centric,  expected  to 
operate  in  ever  increasingly  joint  environments  (DoD,  2004b,  pp.  8, 12-14). 
It  has  become  more  difficult  to  characterize  one  system’s  adequacy  for  its 
own  acquisition  decision  without  interdependent  systems  that  in  and  of 
themselves  are  of  varying  maturity  levels. 


NETWORKED  OPERATIONS 

National  Defense  Strategy  reflects  ever  increasing  multi-Service  and 
Coalition  operations  (DoD,  2008b,  p.  17).  These  self-forming  operations 
preclude  fully  understanding  interfacing  systems  performance  or 
a  Concept  of  Operations  (CONORS)  to  support  an  adequate  OT  in 
advance  of  a  fielding  decision.  While  the  department’s  Testing  in  a 
Joint  Environment  Roadmap  of  2004  set  a  vector  to  lash  together  the 
disparate  testing  capabilities  within  the  department  and  industry,  it 
can  only  go  so  far  given  ever-changing  configuration  baselines  and 
unpredictable  alterations  of  netted  combat  systems  (DoD,  2008b, 
pp.  8,  18).  Testing  and  subsequent  evaluations  will  focus  more  on 
in-theater  assessments. 


EXPANDED  ACQUISITION  AUTHORITY 

Once  the  domain  of  major  commands,  acquisition  authority  has 
spread  to  user-  and  mission-centric  organizations  such  as  Missile 
Defense  Agency  and  Special  Operations  Command,  each  with  its  own 
processes.  As  such,  systems  developers  will  have  less  confidence  in 
their  system’s  performance  as  they  have  less  insight  into,  or  control 
of,  interfacing  systems.  T&E  will  be  less  likely  to  depend  on  a  priori 
knowledge  of  full  system  capability  and  default  to  rudimentary 
baseline  assessments. 


T&E  BEYOND  THE  PROGRAM  MANAGER 

System  complexity  and  interconnectivity  mean  that  testers  will  find  it 
difficult  to  build  test  scenarios  that  characterize  all  desired  performance 
points  within  shorter  development  timeframes.  Added  to  the  difficulty  is  a 
less  well  defined  threat  or  understanding  of  future  CONORS  as  each  user 
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will  likely  tailor  operations  to  their  own  needs.  Acquisition  will  increasingly 
rely  on  tests  outside  the  program  manager’s  control  to  build  just  enough 
insight  for  the  decision  needed.  Capitalizing  on  other  data  from  which 
to  build  consensus  is  key  to  leveraging  integrated  T&E  methodologies  of 
the  revised  DoD  5000.02  (DoD,  2004a;  DoD,  2005,  p.  5;  Defense  Science 
Board,  2008). 


SERVICES  ARE  OVERSIGHT 

Service  T&E  has  undergone  massive  consolidation  since  the  mid-1990s,  a 
situation  that  cannot  readily  be  reversed  in  the  short  term.  Since  2000, 
the  Army  consolidated  much  of  its  T&E  organization  and  reduced  its 
workforce  between  half  and  two-thirds.  The  Navy  reduced  personnel 
and  substantially  integrated  its  prime  contractor/government  testing. 
The  Air  Force  further  shifted  DT  control  to  prime  contractors  with 
commensurate  reductions  in  its  workforce  (Defense  Science  Board, 
2008,  pp.  4-5).  The  burden  of  conducting  traditional  Service  DT  has  fallen 
ever  more  frequently  onto  the  contractors  as  part  of  the  product 
acceptance  process. 

The  above  shows  that  not  only  how  the  department  acquires  its  systems 
has  changed,  but  that  most  cannot  be  addressed  without  fundamental 
change  in  T&E.  Much  of  what  exists  in  today’s  "new”  methodologies  to  help 
nudge  the  acquisition  process  along  in  this  new  environment  ignores  the 
acquisition  and  T&E  world  as  it  exists  today.  The  current  processes  quickly 
succumb  under  the  weight  of  the  endless  reviews  and  forums. 


NO  REAL  CHANCE 

DoD  conducted  three  comprehensive  studies  on  weapons  systems 
acquisition,  with  emphasis  on  T&E,  including  the  Defense  Acquisition 
Performance  Assessment  (DoD,  2005),  the  Defense  Science  Board  Task 
Force  on  Developmental  Test  and  Evaluation  (Defense  Science  Board, 
2008),  and  the  Joint  Defense  Capabilities  Study  (DoD,  2004a).  These 
authoritative  studies  produced  a  myriad  of  recommendations  for  T&E 
and  acquisition.  While  each  report  had  its  emphasis,  their  findings  and 
recommendations  for  T&E  were  generally  similar  and  grouped  into  four 
broad  thrusts: 

1.  Gain  organizational  efficiencies  by  blurring  the  distinction 
between  DT  and  OT. 

2.  Push  discovery  earlier  in  the  process  through  more  rigorous 
testing  up  front. 

Increase  transparency  and  streamline  process  overhead. 


3. 
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4.  Better  utilize  the  planning  and  acquisition  processes  for 
joint  warfighting  needs. 

All  three  studies  presented  strategies  to  efficiently  push  T&E  to  better 
support  the  “buy”  decisions  through  process  streamlining  and  combining 
DT  and  OT  events,  where  possible,  as  part  of  an  integrated  T&E  framework 
feeding  a  continuous  thread  of  discovery.  These  form  the  basis  of  the 
DoD  5000  integrated  T&E  strategy.  Yet,  "integration”  is  fundamentally 
“efficiency”  of  questionable  savings  since  early-on  schedule  and  cost 
avoidance  is  lost  through  later  rework  and  retest. 

Unfortunately,  none  of  the  three  studies  reassessed  T&E’s  role  or 
its  customer.  While  testing  is  fundamental  to  systems  engineering  and 
contractual  compliance,  Acquisition  T&E  is  seen  as  a  “speed  bump”  to 
procurement.  In  trying  to  serve  many  masters,  T&E  became  costly,  less 
efficient,  and  its  reports  of  questionable  utility  to  both  the  buyers  and 
the  users. 


A  FUTURE  OF  MANY  MASTERS 

Future  Acquisition  T&E  must  support  two  acquisition  extremes— the 
quick-reaction,  less  defined  threat  and  the  long-term,  near-peer  threat 
(DoD,  2008b,  pp.  8,  15-17).  It  must  also  support  near-term  contractual 
necessities  as  well  as  longer  term  product  life-cycle  processes.  The 
emphasis  is  early  capability  delivery  for  initial  fielding.  For  many  complex 
systems,  the  department  will  only  begin  to  understand  the  intricacies  and 
capabilities  of  those  systems  once  they  are  delivered  and  operating  in 
the  field.  The  Missile  Defense  Agency  recognized  this  and  developed  T&E 
processes  to  support  dedicated  knowledge  points  that  now  form  the  basis 
for  the  revised  DoD  5000  (Statement,  2008). 


DoD  MUST  BE  MORE  EEEICIENT  AND  EEEECTIVE  IN 
GETTING  INEORMATION  TO  THE  USER  AND  EEEDBACK 
EROM  OPERATIONS  IN  THE  EIELD. 


As  in  WWII,  we  find  ourselves  with  users  forward  deployed,  persistently 
engaged,  and  needing  75  percent  solutions  in  months.  DoD  must  be  more 
efficient  and  effective  in  getting  information  to  the  user  and  feedback  from 
operations  in  the  field. 
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A  New  Model 

Acquisition  T&E  must  focus  on  its  mission,  not  function,  to  support 
acquisition  and  system  complexity  as  they  are  today.  T&E  must  uncover 
critical  risks  prior  to  initiating  a  program  or,  once  begun,  leverage  the 
knowledge  of  subject  matter  experts  as  a  trade-off  against  risk.  The  focus 
today  is  to  provide  capability  as  soon  as  it  is  ready,  with  T&E  the  primary 
mechanism  for  fielding  the  right  capability  at  the  right  time.  This  new  model 
proposes  capability  be  separate  but  affiliated  to  the  buying  decision. 

While  we  acquire  systems  through  the  buying  process,  capabilities 
based  on  aggregates  of  constantly  evolving  systems  are  also  delivered  to 
the  user.  Authority  to  initiate  development  has  become  the  initial  production 
approval  point,  reflecting  the  national  commitment  it  is.  The  acquisition 
process  is  no  longer  the  tidy  affair  it  once  was.  Yet,  it  is  how  DoD  responds 
that  is  the  basis  for  a  new  T&E  model,  which  shifts  emphasis  from  “buying” 
to  the  more  relevant  product  acceptance  and  operational  domains. 


AN  “ALPHA-OMEGA"  MODEL 

The  new  model  for  T&E  shifts  the  emphasis  from  buying  to  two 
basic  but  not  necessarily  sequential  domains:  The  first  domain  is  the 
world  of  acceptance  tests,  or  Alpha  Tests.  Alpha  Tests  are  activities  to 
sufficiently  characterize  systems  in  support  of  contractual  necessities, 
management,  and  initial  fielding  decisions.  The  second  domain— Omega 
Tests— encompasses  the  operational  assessments  that  follow  later,  which 
assess  mission  and  value  added  over  the  fielding  life  cycle. 

The  vision  is  a  T&E  process  that  accelerates  the  delivery  of  initial 
DoD  capability  by  developers,  while  ensuring  continuous  evaluation  of 
performance  in  the  field  for  current  operations  and  future  capability 
development.  This  approach  supports  acquisition  and  life-cycle  activities 
such  as  the  department’s  performance-based  logistics  and  training. 

Alpha  Tests.  Alpha  Tests  are  events  necessary  to  meet  contractual 
requirements  by  capturing  initial  baseline  capability  for  Service  use. 
“Alphas”  comprise  all  initial  experiments,  contractor  development  tests, 
quality  tests.  Service-unique  interface  and  environmental  compliance 
tests,  and  security  and  accreditation  tests,  as  well  as  initial  limited  user 
tests.  They  are  the  necessary  blending  of  Contractor  Tests  used  to 
support  delivery  to  the  government  with  the  traditional  Service-oriented 
interface  testing  (DT)  later  on.  Alpha  Tests  are  a  continuous  aggregate 
of  events,  which  are  not  necessarily  fully  completed  events  or  pass-fail 
by  their  structure.  Alphas  are  agnostic  in  their  management  and  not 
necessarily  under  any  one  single  agent’s  control.  Their  results  form  the 
basis  for  decision  gates,  and  are  ultimately  for  Service  use.  Alpha  Tests 
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provide  the  basis  for  understanding  delivered  items  at  the  time  of  delivery, 
not  necessarily  against  a  priori  baseline  parameters. 

An  Alpha  Test  construct  capitalizes  on  all  existing  datasets,  whether 
or  not  contractor-derived,  and  not  throttled  by  concerns  over  the  color  of 
money,  contracts,  or  ownership.  It  feeds  on  other  Service  efforts,  direct  and 
indirect  program  manager  efforts,  and  training  and  fielding  activities.  Alpha 
fills  the  bin  of  system  knowledge  regardless  of  source.  Further,  an  Alpha 
approach  provides  the  program  manager  and  those  of  affiliated  efforts,  the 
freedom  to  select  the  appropriate  data  from  which  to  argue  the  case  for 
delivery,  up  to  and  including  the  "approval  for  Service  use”  (ASU).  Where 
a  lack  of  data  exists,  the  program  manager  is  obligated  to  fill  the  void  or 
ensure  that  others  do  their  share  to  help  build  the  case  for  ASU. 

An  Alpha  approach  also  requires  involvement  by  customers,  users, 
and  test  and  oversight  agents  for  insight  and  advice,  where  practical 
or  necessary,  given  their  control  over  ASU.  Less  oversight  is  required 
during  Alpha  Test  as  the  burden  falls  on  the  program  manager  to  build  the 
case  necessary  to  deliver  the  incremental  capability  to  the  next  user  or 
integrator  in  the  chain.  This  methodology  is  consistent  with  that  used  by 
sub-tier  vendors  delivering  sub-systems  to  the  LSI  and  consistent  with  the 
department's  Systems  Engineering  Guide  for  Systems  of  Systems  (DoD, 
2008c,  pp.  21,  24-25). 

Omega  Tests.  Omega  Tests  are  those  scripted  and  unscripted,  supervised 
and  un-supervised,  demonstrations  of  systems  operation  in  the  field. 
Users,  OT  agents,  and  oversight,  training,  logistics,  and  doctrine  agents 
focus  on  system  utility  and  are  less  concerned  with  the  buying  decision. 
Omega  Tests  capitalize  on  data  and  experience  in  the  field,  not  to  pass- 
fail  (since  the  department  has  long  since  committed  to  the  program), 
but  to  build  on  the  baseline  understanding  of  capabilities  and  limitations 
at  ASU.  Omega  feedback  also  forms  the  basis  for  the  next  capability 
increment,  or  decision  to  move  on  to  new  capabilities.  Data  and  insight, 
through  formal  reports,  assessments,  or  observations,  are  provided  to 
the  community  at  large,  including  operations  research,  requirements 
generators,  product  life-cycle  managers,  program/project  managers, 
oversight  entities  such  as  Service  Chiefs  and  DOT&E,  and  training  and 
doctrine  agents. 

A  significant  issue,  using  today’s  operational  test  and  evaluation 
construct,  is  pegging  deficiencies  uncovered  in  complex  systems-of- 
systems  tests  for  a  product-centered  acquisition  process.  An  Omega 
strategy  broadens  the  responsibility  as  Omega  Tests  are  funded 
through  a  myriad  of  single  and  combined  sources,  including  component 
commanders;  training  and  doctrine  commands;  research,  logistics,  and 
engineering  activities;  intelligence  agencies;  programs;  and  other  Service 
acquisition  agents. 
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This  approach  expands  the  community  of  Omega  agents  far  beyond 
that  limited  by  the  Service  Operational  Test  Agencies  (OTA)  and  removes 
the  "black  hat”  image  of  today’s  operational  testers.  There  would  be  less 
concern  that  OTA  input  blurs  the  role  between  system  buying  and  fielding. 
Nevertheless,  U.S.C.  Title  10  must  be  revisited  given  the  requirement  to 
conduct  an  OT;  and  for  DOT&E,  an  independent  operational  assessment 
must  be  conducted  prior  to  proceeding  beyond  LRIP.  In  all  likelihood,  DoD 
will  need  consensus  with  Congress  to  either  formalize  a  supervised  period 
of  Alpha  testing  on  basic  systems  to  support  independent  reporting,  or 
use  the  first  Omega  evaluation  as  the  gatekeeper  to  further  cross-Service 
capabilities.  The  latter  would  seem  more  appropriate  as  Congress  and 
the  DoD  get  a  better  picture  of  capabilities  fielded  and  future  needs,  with 
effectiveness  judged  through  a  broader  evaluation  lens. 


ORGANIZING  TO  THE  ALPHA  AND  OMEGA 

Service  field  activities  would  continue  to  function  as  life-cycle  agents 
and  as  centrally  or  direct-funded  Alpha  testers,  supporting  any  Alpha  event 
whether  Service-specific  or  at  contractor  sites.  Alpha,  being  nonpartisan, 
can  be  managed  either  before  formal  program  initiation,  during  program 
phases,  or  as  part  of  post-production  life-cycle  support.  Much  of  this 
structure  is  already  in  place,  as  test  personnel  at  DoD  Major  Range  and  Test 
Facility  Base  activities  are  direct  customer-funded  operations. 

Service  OTAs,  freed  from  the  grip  of  the  acquisition  process,  support 
customers  of  all  types.  OTA  and  Omega  would  be  funded  through  a  much 
broader  array  of  customers  less  tied  to  programs.  The  expeditionary  OTA, 
or  other  agents  tapped  for  such  roles,  deploy  to  theaters  of  operations  or 
specific  test  sites  to  act  as  user  test  or  evaluation  agents.  A  much  smaller 
senior  cadre  would  be  reserved  for  overseeing  Alpha  events  supporting 
ASU  decisions  through  working  arrangements  with  program  offices.  Their 
portfolio  of  products  and  services  would  be  greater  than  current  program¬ 
centric  assessments.  The  OTAs  would  be  managed  by  the  Services,  overseen 
by  DOT&E,  and  free  to  expand  their  operations  worldwide,  including  foreign 
systems.  This  new  and  expanded  role  sets  the  OTA  on  a  path  to  supporting 
a  wider  array  of  warfighting  capabilities. 

The  emphasis  is  on  empowering  with  responsibility  based  on  a  closer 
working  relationship  between  the  developers  and  users.  The  Alpha-Omega 
strategy  relies  on  three  simple  rules  by  which  to  frame  progress  and  argue 
for  ASU,  when  appropriate: 

1.  What  warfighting  capability  is  provided  (not  the  “thing” 
being  procured)? 

2.  To  what  degree  does  it  work,  and  how  do  you  know 
(capabilities/limitations  as  delivered)? 
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3.  What  are  the  impacts  to  other  systems  (risk  assessment 
across  the  doctrine,  organization,  training,  materiel, 
leadership  and  education,  personnel,  and  facilities)? 


EMPOWERED  TESTERS 

Testers  and  evaluators  increase  their  impact  on  new  program  vectors. 
The  emphasis  is  not  on  whether  systems  are  good  enough  to  buy  (as 
they  are  already  being  bought),  but  rather  what  new  vector  must  be  set, 
based  on  performance  and  deficiencies  observed.  OTAs  plan,  manage, 
and  oversee  Omega  Tests  as  well  as  assess  capability  in  the  field,  working 
with  the  users  to  vet  future  capabilities,  upgrades,  or  changes  to  doctrine 
and  CONORS.  A  new  Joint  Omega  Executive  provides  both  independent 
and  collaborative  insight  of  systems-of-systems  operations  in  the  field  to 
support  capability  increments. 


THE  RIGHT  OVERSIGHT 

The  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology 
and  Logistics,  through  either  the  Director  for  Systems  and  Software 
Engineering,  or  the  new  Director  for  Developmental  T&E  (Weapons,  2009) 
oversee  Alpha  activities,  ensuring  that  adequate  insight  and  progress  support 
(along  with  user  representatives)  are  provided  once  capabilities  are  fielded. 
This  lead  would  also  ensure  sufficient  capacity,  training,  and  capability  exist 
at  T&E  facilities.  The  DOT&E  would  oversee  all  Omega  testing,  advise  on 
operational  realism  for  Alpha  events,  and  continue  to  report  independently 
to  Congress.  Oversight  agents  will  focus  less  on  technical  detail,  but 
rather  on  validating  that  achieved  capability  is  usable  and  understood  by 
the  users. 


A  NEW  TEMP 

The  department  has  built  an  entire  cottage  industry  around  the  Test 
and  Evaluation  Master  Plan  (TEMP),  whose  value  is  to  document  the  T&E 
commitment  between  program  manager  and  the  OTA.  Yet,  in  the  pace  of 
today’s  programmatic  change,  the  TEMP  becomes  rapidly  outdated.  Under 
this  process,  the  TEMP  would  not  focus  on  a  program’s  10-year  T&E  plan, 
but  rather  outline  the  top-level  strategy  of  a  shorter  period  leading  to  the 
next  ASU  decision,  focusing  on  system  maturity,  external  resources,  and 
likely  data  collection  points. 
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Conclusions 

The  recently  revised  DoD  5000  strengthens  the  primacy  of  fielding, 
through  acquisition,  with  T&E  primarily  supporting  the  latter.  This  article 
proposed  a  new  map  for  Acquisition  T&E,  supporting  today’s  persistent 
engagement  as  well  as  the  next  near-peer  threat.  The  DoD  cannot  wait 
for  optimal  solutions  before  fielding  capabilities  or  rely  solely  on  T&E  as 
its  gatekeeper.  This  new  Alpha-Omega  strategy,  based  on  acceptance 
testing  for  delivery  and  operational  use  evaluations  in  the  field,  is  on 
par  with  acquisition  as  it  exists  today,  not  on  how  we  wish  it  to  be.  This 
strategy  recognizes  and  accepts  T&E’s  core  role  in  engineering  and  contract 
compliance,  as  well  as  T&E’s  ultimate  customer— the  warfighter. 

This  article  examined  how  the  acquisition  environment  has  changed 
and  how  the  process  itself  has  evolved  as  it  continues  to  adapt  to  this 
new  reality.  Nevertheless,  recent  authoritative  studies  on  T&E  have  not 
recognized  these  fundamental  changes  in  the  landscape  and  have  only 
recommended  modest  changes  to  T&E  processes  to  speed  them  up  a  bit 
and  make  them  cost  a  little  less.  T&E  must  emerge  from  its  relegated  place 
in  the  shadows  of  acquisition  to  support  a  new  customer  set.  The  Alpha- 
Omega  strategy  offers  the  hope  of  changing  this  by  shifting  the  traditional 
OTA  role  out  of  the  “buy”  process  into  the  more  relevant  fielding  process  as 
the  agent  of  choice  for  a  much  wider  set  of  customers,  including  not  only 
Service  acquisition  and  life-cycle  agents,  but  also  component  commanders, 
trainers  and  doctrine  agents,  and  requirements  developers. 

The  Alpha-Omega  Strategy  for  T&E  supports  bringing  capability  to 
the  field  faster,  with  better  understanding  of  capabilities  and  limitations, 
across  a  broader  set  of  systems-of-systems  than  current  methodologies— 
streamlined  or  not— can  ever  do.  The  time  is  right  for  fundamental  change. 
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THE  F119  ENGINE: 

A  SUCCESS  STORY 
OF  HUMAN  SYSTEMS 
INTEGRATION  IN 
AOOUISITION 


}  2ndLt  Kevin  K.  Liu,  USMC,  Ricardo  Vaierdi, 
Donna  H.  Rhodes,  Coi  Larry  Kimm,  USAF,  and 
Lt  Coi  AMs  Headen,  USAF 

The  Department  of  Defense  recently  mandated  the  incorpora¬ 
tion  of  Human  Systems  Integration  (HSI)  early  in  the  acquisition 
cycle  to  improve  system  performance  and  reduce  ownership 
cost.  However,  little  documentation  of  successful  examples  of 
HSI  within  the  context  of  systems  engineering  exists,  making 
it  difficult  for  the  acquisition  community  to  disseminate  and 
apply  best  practices.  This  article  presents  a  case  study  of  a 
large  Air  Force  project  that  represents  a  successful  applica¬ 
tion  of  HSI.  The  authors  explore  the  influence  of  both  the  Air 
Force  and  the  project  contractor.  Additionally,  they  identify 
top-level  leadership  support  for  integrating  HSI  into  systems 
engineering  processes  as  key  to  HSI  success,  reinforcing  the 
importance  of  treating  HSI  as  an  integral  part  of  pre-Milestone 
A  activities. 


Keywords:  Human  Systems  Integration,  Systems 
Engineering,  Acquisition  Process,  Human  Factors 
Engineering,  integrated  Product  Development 
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Human  Systems  Integration  (HSI)  is  defined  as  the  “interdisciplinary 
technical  and  management  processes  for  integrating  human  considerations 
within  and  across  all  system  elements;  an  essential  enabler  to  systems 
engineering  practice”  (Haskins,  2007).  The  primary  objective  of  HSI  is  to 
integrate  the  human  as  a  critical  system  element,  regardless  of  whether 
humans  in  the  system  function  as  individuals,  teams,  or  organizations.  The 
discipline  seeks  to  treat  humans  as  equally  important  to  system  design  as 
are  other  system  elements,  such  as  hardware  and  software. 

Many  stakeholders  have  attempted  to  define  HSI,  and  the  number  and 
definitions  of  HSI  domains  vary  by  organization  (Department  of  Defense 
[DoD],  2008).  However,  DoD  guidance  makes  it  clear  that  the  ultimate 
goal  of  any  HSI  program  should  be  to  “optimize  total  system  performance, 
minimize  total  ownership  costs,  and  ensure  that  the  system  is  built  to 
accommodate  the  characteristics  of  the  user  population  that  will  operate, 
maintain,  and  support  the  system”  (DoD,  2008;  Department  of  the  Army, 
2005;  Department  of  the  Navy,  2008).  Since  this  article  documents  a  case 
study  of  HSI  practice  within  the  Air  Force,  we  provide  the  nine  domains  of 
HSI,  as  highlighted  on  the  previous  page. 

Large  defense  projects  require  significant  systems  engineering 
effort  that  can  quickly  drive  up  costs.  At  the  same  time,  defense  projects 
typically  have  high  requirements  for  survivability,  safety,  and  other  human 
considerations.  As  mentioned  earlier,  DoD  is  interested  in  HSI  as  a  means 
of  both  reducing  cost  (Wallace  et  al.,  2007),  shortening  acquisition 
cycles  (Mack  et  al.,  2007),  and  improving  system  performance  (DoD, 
2008).  Published  case  studies  and  best  practices  have  highlighted  the 
technical  and  economic  benefits  of  successful  HSI  practice  (Booher, 
1997;  Landsburg  et  al.,  2008).  These  studies  and  others  have  consistently 
emphasized  the  importance  of  taking  HSI  into  consideration  early  in  the 
acquisition  process. 

Although  HSI  evolved  from  the  study  of  Human  Factors,  it  expands 
upon  the  latter  discipline  by  incorporating  a  broader  range  of  human 
considerations  such  as  occupational  health,  training,  and  survivability  over 
the  system  life  cycle.  Depending  on  the  particular  definitions  being  used, 
the  areas  covered  by  Human  Factors  and  HSI  can  overlap.  The  best  way  to 
understand  the  differences  between  the  two  terms  is  that  HSI  is  at  heart 
a  subset  of  systems  engineering.  HSI  work  must  take  place  in  conjunction 
with  systems  engineering  and  applies  to  all  the  same  acquisition  phases. 
Historically,  many  engineers  have  tended  to  view  human  factors  (and 
therefore  HSI)  as  a  means  of  identifying  problems  with  a  design,  rather 
than  as  an  enabler  of  good  design  (Harrison  &  Forster,  2003).  Although 
HSI  analyses  in  the  later  phases  of  acquisition  are  an  important  part  of  HSI 
success,  the  case  study  presented  in  this  article  focuses  on  the  role  and 
impacts  of  HSI  in  systems  engineering  throughout  the  acquisition  life  cycle. 


The  F119  Engine:  A  Success  Story  of 
Human  Systems  Integration  in  Acquisition 


Method 

This  case  study  documents  HSI  activities  during  the  development  of 
Pratt  &  Whitney's  F119  engine,  which  powers  the  $143  million  Lockheed 
Martin  F-22  Raptor  fighter  aircraft  (Drew,  2008).  The  F-22  Raptor  (Figure  1) 
fulfils  the  air  superiority  role  in  the  Air  Force  by  using  a  package  of 
technologies  to  allow  pilots  to  “track,  identify,  shoot,  and  kill  air-to-air 
threats  before  being  detected”  (Department  of  the  Air  Force,  2008b). 
Although  the  Air  Force  HSI  Office  was  not  formalized  until  2007,  much  of 
the  work  done  on  the  F-22  and  F119  in  the  1980s  and  1990s  spans  the 
domains  of  HSI,  making  the  F119  a  best  practice  of  HSI  in  the  Air  Force. 

In  designing  the  study,  we  followed  Yin’s  (2003)  approach  for 
identifying  five  important  components  to  case  study  design:  1)  a  study's 
questions,  2)  its  proposition,  3)  its  units  of  analysis,  4)  the  logic  linking  the 
data  to  the  propositions,  and  5)  the  criteria  for  interpreting  the  findings. 


FIGURE  1.  THE  F-22  RAPTOR 


Flying  High.  The  Lockheed  Martin/Boeing  F-22  Raptor  is  a  fifth-generation  fighter 
aircraft  that  uses  stealth  technology.  It  was  designed  primarily  as  an  air  superiority 
fighter,  but  has  additional  capabilities  that  include  ground  attack,  electronic  warfare, 
and  signals  intelligence  roles.  The  Raptor  was  first  introduced  into  the  U.S.  Air  Force  in 
December  2005.  Retrieved  2009  from  Inside  AF.mil  [Web  page]  at  http://www.af.mil/ 
shared/media/photodb/photos/090123-F-2828D-942.JPG.  U.S.  Air  Force  photo  by 
Air  Force  Master  Sgt  A.  Dunaway  (2008) 
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The  case  study  was  designed  around  three  central  research  questions: 

1.  How  did  Pratt  &  Whitney  determine  how  much  HSI  effort 
would  be  needed? 

2.  How  much  did  HSI  effort  eventually  cost? 

3.  How  did  HSI  fit  into  the  larger  systems  engineering  picture? 

The  first  two  of  our  research  questions  reflect  our  ongoing  research  on 
the  economics  of  HSI.  Discussion  of  this  case  study  from  the  perspective  of 
cost  estimation  can  be  found  in  Liu,  Valerdi,  and  Rhodes  (2009),  and  Valerdi 
and  Liu  (2009).  In  this  article,  we  address  the  third  research  question. 

Since  we  sought  to  describe  how  the  F119  became  a  best  practice 
of  HSI,  we  designed  our  study  as  a  single-case  descriptive  study.  Our 
proposition  was  that  HSI  effort  could  be  isolated  from  the  larger  systems 
engineering  effort  spent.  Initially,  we  hoped  to  establish  a  quantitative 
relationship  between  HSI  cost  and  systems  engineering  cost,  but  were 
also  interested  in  identifying  the  critical  factors  that  led  to  successful  HSI 
implementation.  Although  our  third  research  question  was  originally  meant 
to  better  our  understanding  of  HSI  cost,  we  found  in  the  course  of  our  case 
study  that  the  role  of  HSI  in  systems  engineering  is  not  well  understood  and 
would  benefit  from  the  documentation  of  a  best  practice. 

We  sought  to  analyze  the  early  development  of  the  F119,  from  concept 
development  until  major  engineering  and  manufacturing  development 
(EMD).  Our  unit  of  analysis  was  the  engineering  organization  responsible 
for  HSI  on  the  F119  at  Pratt  &  Whitney.  Since  historical  data  on  specific 
costs  associated  with  HSI  activities  were  not  available  either  because  data 
were  not  kept  or  the  records  could  not  be  found,  we  depended  on  Pratt  & 
Whitney  employees  familiar  with  the  F119  to  build  an  understanding  of  its 
development.  We  conducted  a  series  of  interviews  with  Pratt  &  Whitney 
engineers  who  were  active  in  the  development  of  the  F119,  in  both  technical 
and  management  roles.  Based  on  our  central  proposition  and  research 
questions,  our  interviews  focused  both  on  life-cycle  cost  measurement  as 
well  as  on  systems  engineering  and  HSI  methodology.  With  this  information, 
we  were  able  to  identify  key  HSI  success  factors.  We  concluded  the  case 
study  by  validating  our  results  using  existing  literature  on  the  F119  and  the 
F-22  and  by  comparing  the  results  of  our  interviews  with  multiple  engineers. 


EARLY  AIR  FORCE  EMPHASIS  ON  RELIABILITY  AND  MAINTAINABILITY 

The  Defense  Resources  Board  approved  the  creation  of  the  Advanced 
Tactical  Fighter  (ATF)  program  in  November  of  1981  to  create  a  military  jet 
that  would  be  able  to  guarantee  air  superiority  against  the  Soviet  Union. 
This  fighter  was  meant  to  replace  the  F-15  Eagle,  which  had  previously  filled 
this  role.  A  team  composed  of  Lockheed,  Boeing,  and  General  Dynamics 
competed  against  Northrop  Grumman  to  develop  the  fighter.  In  1991,  the 
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ATF  contract  was  awarded  to  the  Lockheed  team’s  F-22,  powered  by  Pratt 
&  Whitney’s  F119  engine  (Figure  2).  Then-Secretary  of  the  Air  Force  Donald 
Rice  noted  that  an  important  consideration  in  the  awarding  of  the  contract 
was  the  fact  that  the  F-22’s  engines  offered  superior  reliability  and 
maintainability  (Bolkcom,  2007). 

FIGURE  2.  F119  ENGINE 


# 

/ 


Cutaway  of  Pratt  &  Whitney’s  (P&W)  F119-PW-100  engine.  Two  F119-PW-100  engines 
power  the  Lockheed  Martin  F-22  Raptor,  the  U.S.  Air  Force's  new  stealth  fighter.  The 
F119  features  a  unique  thrust  vectoring  nozzle,  integrated  stealth  characteristics,  and 
the  capability  to  supercruise,  or  achieve  Mach  1.5  without  afterburner.  Adapted  from 
“Pratt  &  Whitney’s  F119  Engine  Receives  ISR  Approval  from  USAF,  Surpasses  4,000 
Flight  Hours,  Demonstrates  Unprecedented  Reliability,”  Pratt  &  Whitney  Press  Release, 
September  16,  2002.  Retrieved  2009  from  http://www.pw.utc. com/Media+Center/ 
Press+Releases/Pratt+&+Whitney's+F119+Engine+Receives+ISR+Approval+from+US 
AF,+Surpasses+4,000+Flight+Hours,+Demonstrates+Unprecedented+ Reliability. 

The  Air  Force  placed  an  emphasis  on  reliability  and  maintainability 
from  the  beginning  of  the  ATF  program  as  well  as  throughout  the  Joint 
Advanced  Fighter  Engine  program  (JAFE)— the  program  to  develop  the 
engine  for  the  ATF.  In  June  of  1983,  four  general  officers  representing  the 
Army,  Navy,  and  Air  Force  signed  a  joint  agreement  in  order  to  “emphasize 
to  the  DoD  and  defense  contractor  communities  the  critical  importance 
of  improving  operational  system  availability  by  making  weapon  system 
readiness  and  support  enhancement  high-priority  areas  for  all  our  research 
and  development  activities”  (Keith  et  al.,  1983).  Later  that  year,  the  director 
of  the  JAFE  program  sent  a  memorandum  to  participants  in  the  program, 
including  Pratt  &  Whitney,  asking  them  to  consider  that  over  50  percent  of 
the  Air  Force  budget  was  then  devoted  to  logistics,  and  that  the  problem 
would  only  worsen  (Reynolds,  1983). 


April  2010  I  2  8  9 


2  9  0 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


To  address  this  increase  in  logistics  cost  and  determine  ways  to  develop 
creative  solutions,  the  Air  Force  created  the  Reliability,  Maintainability,  & 
Sustainability  (RM&S)  program  in  1984  (Gillette,  1994).  Besides  reducing 
life-cycle  cost,  the  RM&S  program  also  sought  to  address  the  reliability 
and  durability  problems  that  had  plagued  Pratt  &  Whitney’s  previous 
FlOO  engine,  which  powered  the  Air  Force’s  F-15  Eagle.  Developed  in  the 
1970s,  the  F-15  was  developed  specifically  to  counter  the  Russian  MiG-25. 
Therefore,  emphasis  was  placed  on  performance  during  the  development 
of  both  the  F-15  and  FlOO.  Unfortunately,  the  high  performance  of  the 
FlOO  meant  that  the  engine  was  more  prone  to  failure  and  downtime.  By 
the  1980s,  the  Russian  air  superiority  threat  was  no  longer  as  pressing 
as  when  the  F-15  was  developed,  and  supportability  was  emphasized 
over  performance.  As  a  result,  the  Air  Force  wanted  improved  RM&S 
not  only  on  the  F119  engine,  but  on  development  of  the  F-22  as  a  whole. 
Specific  supportability  goals  for  the  F-22  were  announced  as  early  as  1983 
(Aronstein  et  al.,  1998). 


UNDERSTANDING  CUSTOMER  NEEDS 

The  F-22  engine  competition  was  not  the  only  instance  in  which 
Pratt  &  Whitney  had  competed  with  General  Electric.  Both  companies 
had  developed  engines  to  power  the  Air  Force’s  F-16  Fighting  Falcon. 
In  the  end.  General  Electric  provided  the  majority  of  engines  for  that 
platform.  Pratt  &  Whitney  saw  success  in  the  JAFE  program  as  critical 
to  the  company’s  ability  to  continue  to  compete  in  the  military  engine 
market.  For  the  F119  engine,  Pratt  &  Whitney  decided  not  only  to  meet 
the  Air  Force’s  RM&S  requirements,  but  to  emphasize  designing  for  the 
maintainer  throughout  all  aspects  of  the  program.  The  company’s  approach 
exemplified  the  best  practices  of  what  is  now  known  as  HSI. 

Pratt  &  Whitney  conducted  approximately  200  trade  studies  as 
contracted  deliverables  for  the  Air  Force.  Pratt  &  Whitney  engineers  also 
estimated  they  had  conducted  thousands  of  informal  trade  studies  for 
internal  use.  These  trade  studies  used  evaluation  criteria,  including  safety; 
supportability;  reliability,  maintainability,  operability,  and  stability;  and 
manpower,  personnel,  and  training  (Deskin  &  Yankel,  2002). 

Figures  of  merit  were  developed  for  the  trade  studies  to  define  a 
consistent  set  of  criteria  upon  which  to  assess  the  trade  studies.  Pratt 
&  Whitney  engineers  used  these  figures  of  merit  to  determine  which 
engineering  groups  would  participate  in  each  trade  study. 

As  is  often  the  case  in  the  development  of  complex  defense  systems, 
responsibilities  for  the  various  domains  of  HSI  are  distributed  among 
many  different  organizations  at  Pratt  &  Whitney.  Of  the  nine  domains  of 
HSI  (see  Table),  seven  were  represented  in  Pratt  &  Whitney’s  engineering 
groups.  Maintainability,  Survivability,  Safety,  Training,  and  Materials  were  all 
engineering  groups  at  Pratt  &  Whitney.  Manpower,  Personnel,  and  Human 
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TABLE.  NINE  DOMAINS  OF  HUMAN  SYSTEMS  INTEGRATION 


Manpower 

The  number  and  mix  of  personnel  (military,  civilian,  and 
contractor)  authorized  and  available  to  train,  operate, 
maintain,  and  support  each  system. 

Personnel 

The  human  aptitudes,  skills,  knovyledge,  experience  levels, 
and  abilities  required  to  operate,  maintain,  and  support  a 
system  at  the  time  it  is  fielded. 

Training 

The  instruction  and  resources  required  to  provide 
personnel  vyith  requisite  knovyledge,  skills,  and  abilities  to 
properly  operate,  maintain,  and  support  a  system. 

Environment 

In  the  context  of  HSI,  the  conditions  in  and  around  the 
system  and  the  concepts  of  operation  that  affect  the 
human’s  ability  to  function  as  a  part  of  the  system,  as  well 
as  the  requirements  necessary  to  protect  the  system  from 
the  environment  (e.g.,  radiation,  temperature,  acceleration 
forces,  all-weather  ops,  day-night  ops,  laser  exposure,  air 
quality  within  and  around  the  system,  etc.). 

Safety 

The  application  of  systems  engineering  and  systems 
management  in  conducting  hazard,  safety,  and  risk 
analysis  in  system  design  and  development  to  ensure 
that  all  systems,  subsystems,  and  their  interfaces  operate 
effectively,  without  sustaining  failures  or  jeopardizing  the 
safety  and  health  of  operators,  maintainers,  and  the 
system  mission. 

Occupational 

Health 

The  consideration  of  design  features  that  minimize  risk 
of  injury,  acute  and/or  chronic  illness  or  disability,  and/or 
that  reduce  job  performance  of  personnel  who  operate, 
maintain,  or  support  the  system. 

Habitability 

Factors  of  living  and  working  conditions  that  are  necessary 
to  sustain  the  morale,  safety,  health,  and  comfort  of  the 
user  population  that  contribute  directly  to  personnel 
effectiveness  and  mission  accomplishment,  and  often 
preclude  recruitment  and  retention  problems. 

Survivability 

The  ability  of  a  system,  including  its  operators,  maintainers, 
and  sustainers,  to  withstand  the  risk  of  damage,  injury,  loss 
of  mission  capability,  or  destruction. 

Human 

Factors 

Engineering 

The  comprehensive  integration  of  human  capabilities 
and  limitations  (cognitive,  physical,  sensory,  and  team 
dynamics)  into  systems  design  to  optimize  human 
interfaces  and  facilitate  human  performance  in  training, 
operation,  maintenance,  support,  and  sustainment  of 
a  system. 
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Factors  Engineering  were  taken  into  account  by  the  Maintainability  group. 
Human  Factors  Engineering  also  impacted  the  Safety  group.  Occupational 
Health  was  considered  by  both  the  Safety  group  and  Materials  group, 
which  dealt  with  hazardous  materials  as  one  of  its  responsibilities.  While 
there  was  an  Environmental  Health  and  Safety  (EH&S)  group  at  Pratt 
&  Whitney,  it  dealt  with  EH&S  within  the  organization  itself  and  did  not 
impact  engine  design.  Habitability  was  not  an  important  consideration  in 
the  engine  design. 


TOP-LEVEL  LEADERSHIP  AND  INTEGRATED  PRODUCT  DEVELOPMENT 

The  major  requirements  for  RM&S  came  directly  from  the  Air  Force.  The 
JAFE  program  in  particular  was  intended  to  improve  RM&S  by  “reducing 
the  parts  count,  eliminating  maintenance  nuisances  such  as  safety  wire, 
reducing  special-use  tools,  using  common  fasteners,  improving  durability, 
improving  diagnostics,  etc.”  (Aronstein  et  al.,  1998).  While  General 
Electric  made  significant  RM&S  improvements  to  its  F120  engine  during 
this  time  period,  Pratt  &  Whitney  centered  its  competitive  strategy  on 
RM&S  superiority. 

During  the  Joint  Advanced  Fighter  Engine  competition,  Pratt  &  Whitney 
participated  in  the  Air  Force’s  “Blue  Two”  program.  The  name  refers  to  the 
involvement  of  maintenance  workers  in  the  Air  Force— “blue-suiters.”  The 
program  brought  Pratt  &  Whitney  engineers  to  Air  Force  maintenance 
facilities  so  that  the  engine  designers  could  experience  first-hand  the 
challenges  created  for  maintainers  by  their  designs.  Maintainers  showed 
how  tools  were  poorly  designed,  manuals  had  unclear  instructions,  and  jobs 
supposedly  meant  for  one  person  took  two  or  more  to  complete  safely. 

Many  of  the  features  for  which  the  F119  would  come  to  be  praised  were 
a  result  of  leadership  commitment  to  HSI.  Frank  Gillette,  the  Chief  Engineer 
of  the  F119,  served  in  various  leadership  positions  on  the  F119  project, 
eventually  leading  a  team  of  over  900  engineers.  In  interviews  with  Pratt  & 
Whitney  employees  familiar  with  the  F119,  Gillette  was  identified  as  a  driving 
force  behind  ensuring  buy-in  to  HSI  principles. 

When  the  Pratt  &  Whitney  team  returned  from  its  Blue  Two  experience 
to  work  on  the  F119,  Gillette  captured  the  lessons  learned  from  the  site  visits 
in  a  series  of  presentations.  These  presentations  were  then  shown  to  every 
engineer  on  the  F119  team.  Gillette  also  established  design  ground  rules 
based  on  the  requirements  of  the  maintainer. 

One  of  the  most  important  requirements  for  the  F119  was  that  only  five 
hand  tools  should  be  used  to  service  the  entire  engine.  All  Line  Replaceable 
Units  (LRUs)  would  have  to  be  “one-deep,”  meaning  that  the  engine  would 
have  to  be  serviceable  without  removal  of  any  other  LRUs,  and  each  LRU 
would  have  to  be  removable  using  a  single  tool  within  a  20-minute  window 
(Gillette,  1994).  Maintenance  would  have  to  be  possible  while  wearing 
hazardous  environment  protection  clothing.  Maintenance  tasks  would 
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have  to  accommodate  the  heights  of  maintainers  from  the  5th  percentile 
female  to  the  95th  percentile  male+  (Gillette,  1994;  Aronstein  et  al.,  1998). 
In  addition: 

Built-in  test  and  diagnostics  were  integrated  with  the  aircraft 
support  system,  eliminating  the  need  for  a  special  engine  support 
system.  Lockwire  was  eliminated,  and  torque  wrenches  were  no 
longer  required  for  “B”  nut  installations.  The  engine  was  designed 
with  built-in  threadless  borescope  ports,  axially  split  cases,  oil 
sight  gauges,  and  integrated  diagnostics.  Other  improvements 
were  a  modular  design. ..color-coded  harnesses,  interchangeable 
components,  quick  disconnects,  automated  integrated  maintenance 
system,  no  component  rigging,  no  trim  required,  computer-based 
training,  electronic  technical  orders,  and  foreign  object  damage 
and  corrosion-resistant.  These  advances  were  intended  to  reduce 
operational-level  and  intermediate-level  maintenance  items  by  75 
percent  and  depot-level  tools  by  60  percent,  with  a  40  percent 
reduction  in  average  tool  weight.  (Aronstein  et  al.,  1998) 

These  innovations  were  only  possible  using  the  Integrated  Product 
Development  (IPD)  concept.  Whereas  on  previous  projects,  engineering 
groups  at  Pratt  &  Whitney  each  worked  in  their  own  respective  disciplines, 
under  IPD  teams  of  engineers  from  varying  disciplines  were  able  to  provide 
design  engineers  with  the  perspectives  they  needed  to  see  the  full  impacts 
of  their  design  decisions. 


CONTINUING  ACCOUNTABILITY  AND  ENFORCEMENT  OFHSI 

Adoption  of  the  IPD  concept  brought  various  stakeholders  together 
early  in  the  design  process  and  ensured  multidisciplinary  input  through 
design  and  development.  As  a  matter  of  policy,  whenever  a  design  change 
needed  to  be  made,  the  originating  group  would  submit  the  change  to  be 
reviewed  by  a  Configuration  Control  Board  (CCB).  CCBs  were  composed 
of  senior  engineers  from  multiple  engineering  groups.  At  CCB  meetings, 
each  group  with  a  stake  in  a  particular  design  change  would  explain  the 
impacts  of  that  change  to  the  chair  of  the  CCB,  typically  a  design  engineer. 
The  chair  would  then  weigh  the  different  considerations  of  the  design 
change  and  either  approve/disapprove  the  change  or  recommend  further 
analysis  be  done. 

In  instances  when  Air  Force  requirements  needed  to  be  changed,  the 
originating  group  would  submit  a  Component  Integration  Change  Request 
(CICR),  which  would  then  be  internally  debated  much  as  with  design 
changes.  CICRs  were  typically  initiated  when  it  was  determined  that  a 
particular  requirement  might  not  be  in  the  best  interests  of  the  customer  or 
when  one  requirement  conflicted  with  another.  Once  a  CICR  was  finalized 


294 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


internally  by  all  of  Pratt  &  Whitney’s  engineering  groups,  it  was  presented 
to  the  Air  Force,  which  would  then  make  the  final  decision  on  whether  a 
requirement  could  be  eliminated,  modified,  or  waived. 

The  processes  for  design  and  requirement  change  ensured  that 
the  work  of  one  group  did  not  create  unforeseen  problems  for  another. 
However,  change  requests  were  typically  made  in  response  to  problems 
that  arose  during  development.  Although  reacting  to  and  fixing  these 
problems  were  important,  it  took  proactive  leadership  to  make  sure  HSI 
principles  were  being  followed  even  when  no  problems  were  apparent. 

Frank  Gillette  created  several  policies  that  ensured  engineers  kept 
RM&S  considerations  constantly  in  mind.  All  part  design  drawings  were 
required  to  be  annotated  with  the  tools  needed  to  service  that  part.  This 
helped  to  achieve  the  goal  of  being  able  to  service  the  entire  engine  with 
only  five  hand  tools  (in  the  end,  the  F119  required  five  two-sided  hand  tools 
and  one  other  tool,  sometimes  described  as  11  tools  total). 

Gillette  also  insisted  on  the  development  of  several  full-scale  mock-ups 
of  the  F119.  These  mock-ups  came  at  a  considerable  cost  (over  $2  million 
each,  while  the  cost  of  an  engine  was  then  about  $7  million)  but  allowed 
engineers  to  see  whether  their  designs  had  really  achieved  maintainability 
goals.  Engineers  were  asked  to  service  LRUs  on  the  mock-ups  by  hand  to 
ensure  that  they  were  each  indeed  only  "one-deep.”  When  an  LRU  was 
shown  to  not  meet  that  requirement,  the  teams  responsible  for  those  LRUs 
were  asked  to  redesign  them. 


HSI  EFFORTS  LEAD  TO  COMPETITION  SUCCESS 

Leading  up  to  the  major  EMD  contracts  awarded  in  1991,  Pratt  & 
Whitney  conducted  400  distinct  demonstrations  of  the  F119’s  RM&S 
features.  The  F119  also  accrued  over  110,000  hours  of  component  tests 
and  3,000  hours  of  full-up  engine  tests,  representing  a  thirtyfold  increase 
in  total  test  hours  over  its  predecessor,  the  FlOO  (Aronstein  et  al.,  1998). 
Pratt  &  Whitney  was  willing  to  spend  significant  effort  on  demonstrating 
the  F119’s  RM&S  features  because  the  company  had  recently  been  beat 
out  by  General  Electric  in  their  competition  to  provide  engines  for  the  Air 
Force’s  F-16  Fighting  Falcon,  and  therefore  saw  the  Joint  Advanced  Fighter 
Engine  competition  as  its  last  chance  to  stay  in  the  military  engine  market. 

In  1991,  both  Pratt  &  Whitney  and  General  Electric  were  awarded 
contracts  worth  $290  million  to  complete  the  EMD  phase  of  competition. 
The  companies  were  given  independence  as  to  the  number  and  types  of 
tests  that  would  be  run  on  their  engines,  while  the  Air  Force  provided  safety 
oversight.  As  a  result,  Pratt  &  Whitney  chose  to  log  about  50  percent  more 
test  hours  than  General  Electric  (Aronstein  et  al.,  1998). 

General  Electric  chose  to  emphasize  the  performance  of  its  F120  engine 
over  RM&S,  though  the  F120  did  meet  the  Air  Force’s  RM&S  requirements. 
The  F120  was  the  world’s  first  flyable  variable  cycle  engine  (Hasselrot  & 
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Montgomerie,  2005).  This  meant  that  the  F120  was  able  to  change  from 
turbofan  to  turbojet  configuration  to  achieve  maximum  performance  in 
multiple  flight  situations.  The  F120  was  tested  in  both  Lockheed’s  YF-22  and 
Northrop  Grumman’s  YF-23  prototypes,  demonstrating  better  maximum 
speed  and  supercruise  than  Pratt  &  Whitney’s  F119  in  both  cases  (Aronstein 
et  al.,  1998).  The  dry  weight  of  the  F119  is  classified,  making  it  impossible 
to  calculate  its  exact  thrust-to-weight  ratio.  Flowever,  Pratt  &  Whitney 
advertises  the  F119  as  a  35,000-lb  thrust  class  engine,  putting  it  into  the 
same  thrust  class  as  the  F120  (Gunston,  2007). 

Despite  the  F120’s  superior  performance  in  the  air  and  higher  thrust-to- 
weight  ratio,  on  April  23, 1991,  the  Air  Force  chose  the  combination  of  Pratt 
&  Whitney’s  F119  and  Lockheed’s  YF-22  to  be  developed  into  the  F-22.  Pratt 
&  Whitney  had  repeatedly  demonstrated  a  better  understanding  of  the  Air 
Force’s  RM&S  needs,  investing  more  time  and  money  into  demonstrations 
and  internal  efforts  than  its  competitor.  It  also  avoided  the  increased  risk 
of  developing  a  variable  cycle  engine,  at  the  time  considered  a  relatively 
new  and  untested  technology.  By  1991,  the  Air  Force’s  RM&S  program  was 
less  focused  on  reducing  downtime  and  more  concerned  with  reducing 
life-cycle  costs.  Pratt  &  Whitney  had  presented  a  management  plan  and 
development  schedule  that  the  Air  Force  considered  sensitive  to  their  needs 
(Aronstein  et  al.,  1998).  On  August  2, 1991,  contracts  worth  $11  billion  were 
awarded  to  Lockheed  and  Pratt  &  Whitney  (Bolkcom,  2007),  demonstrating 
the  Air  Force’s  commitment  to  HSI.  Pratt  &  Whitney’s  portion  was  worth 
$1,375  billion  alone  (Aronstein  et  al.,  1998). 


KEY  HSI  SUCCESS  FACTORS 

The  Air  Force’s  early  and  continuing  emphasis  on  RM&S  was  captured 
via  requirements.  Although  dating  back  to  2003  the  General  Accounting 
Office  (GAO,  now  the  Government  Accountability  Office)  was  still 
advocating  for  more  equal  consideration  of  reliability  and  maintainability 
in  requirements  definition  (GAO,  2003),  our  case  study  showed  that  the 
Air  Force  had  already  understood  this  principle  a  decade  prior.  The  Air 
Force’s  initial  guidance  to  emphasize  RM&S  shaped  the  design  approach 
of  all  of  its  contractors. 

The  actions  of  both  the  Air  Force  and  Pratt  &  Whitney  were  examples  of 
combining  top-level  leadership's  role  within  systems  engineering  practices. 
The  Air  Force  set  formal  requirements  and  expected  deliverable  trade 
studies,  but  it  also  set  early  supportability  goals,  released  memoranda 
explaining  their  intent,  and  funded  programs  to  show  Pratt  &  Whitney 
engineers  actual  maintenance  conditions.  In  its  own  right,  Pratt  &  Whitney 
embraced  the  IPD  approach  along  with  IPD’s  subordinate  systems 
engineering  processes,  but  also  invested  significant  effort  to  develop 
mock-ups,  conduct  additional  testing,  and  hold  engineers  accountable  for 
RM&S  standards. 
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As  a  result,  we  identify  three  factors  as  key  to  the  success  of  HSI  in  the 
context  of  systems  engineering  in  the  F119  program: 

1.  Air  Force  policy  to  elevate  the  visibility  of  FISI 

2.  Pratt  &  Whitney's  willingness  to  internalize  HSI  practices 
and  enforce  accountability  for  HSI 

3.  The  integration  of  HSI  and  systems  engineering  in  the  early 
phases  of  the  acquisition  life  cycle. 

Conclusions 

In  this  case  study,  we  document  an  example  of  successful  HSI.  HSI 
strongly  influenced  the  development  of  Pratt  &  Whitney’s  F119  turbofan 
engine  from  early  in  the  acquisition  life  cycle  through  EMD. 

Many  traditional  systems  engineering  activities  also  were  clearly 
impacted.  Conversations  with  Pratt  &  Whitney  engineers  indicated  that 
by  the  time  HSI  requirements  were  integrated  into  the  engine,  the  cost 
of  specific  HSI  activities  could  no  longer  be  distinguished  from  other 
systems  engineering  costs.  In  addition,  Pratt  &  Whitney  never  had  a  formal 
organization  responsible  for  all  HSI  considerations.  Instead,  responsibilities 
for  HSI  were  spread  between  multiple  engineering  groups.  The  lack  of  a 
formal  HSI  group  did  not  prevent  the  F119  from  becoming  a  best  practice  of 
HSI.  To  the  contrary,  the  fact  that  HSI  considerations  were  tightly  coupled  to 
other  systems  engineering  practices  was  one  the  project’s  major  strengths. 

This  case  study  represents  a  first  step  toward  establishing  the  role 
of  HSI  in  the  context  of  systems  engineering.  As  more  success  stories 
are  documented,  the  ability  to  disseminate  best  practices  throughout 
the  defense  acquisition  community  will  improve  and  will  lead  to  reduced 
lifecycle  costs  and  improved  performance. 


The  F119  Engine:  A  Success  Story  of 
Human  Systems  Integration  in  Acquisition 


ACKNOWLEDGMENTS 

The  authors  gratefully  acknowledge  funding  for  this  research  provided 
through  the  U.S.  Air  Force  Human  Systems  Integration  Office,  MIT  Systems 
Engineering  Advancement  Research  Initiative  (SEAri,  http://seari.mit. 
edu),  and  Lean  Advancement  Initiative  (LAI,  http://lean.mit.edu).  The 
authors  would  also  like  to  thank  the  employees  at  Pratt  &  Whitney  for  their 
participation  in  the  case  study. 

The  work  presented  in  this  article  is  part  of  a  larger  ongoing  project 
to  understand  the  economics  of  Human  Systems  Integration,  focusing  on 
cost  estimation  approaches  and  leading  indicators.  The  authors  have  also 
produced  a  series  of  conference  papers  and  a  book  chapter  on  this  topic, 
which  are  cited  in  the  reference  section. 


April  2010  I  2  9  7 


2  9  8 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mi 


Author  Biographies 


2ndLt  Kevin  Liu,  USMC,  is  a  graduate  research 
assistant  at  the  Systems  Engineering 
Advancement  Research  Initiative  at  the 
Massachusetts  Institute  of  Technology  (MIT). 
He  is  currently  pursuing  a  Master  of  Science  in 
Technology  and  Policy  from  MIT  and  received 
a  Bachelor  of  Science  in  Electrical  Engineering 
from  the  U.S.  Naval  Academy.  His  research 
interests  include  cost  estimation  and  Human 
Systems  Integration. 

(E-mail  address:  k_liu@mit.edu) 

Dr.  Ricardo  Valerdi  is  a  research  associate 
at  the  Lean  Advancement  Initiative  at  the 
Massachusetts  Institute  of  Technology.  He 
holds  a  Doctor  of  Philosophy  in  Industrial 
and  Systems  Engineering  from  the  University 
of  Southern  California.  Dr.  Valerdi’s  research 
interests  are  in  systems  engineering,  cost 
estimation,  decision  making,  enterprise 
transformation,  and  enterprise  performance 
measurement.  He  is  on  the  Board  of 
Directors  of  the  International  Council  on 
Systems  Engineering  and  is  the  founding 
CO  -  Ed  i  to  r- i  n  -  C  h  i  ef  of  the  Journal  of 
Enterprise  Transformation. 


(E-mail  address:  rvalerdi@mit.edu) 


The  F119  Engine:  A  Success  Story  of 
Human  Systems  Integration  in  Acquisition 


April  2010  I  2  9  9 


Dr.  Donna  H.  Rhodes  is  a  senior  lecturer  and 
principal  research  scientist  in  the 
Massachusetts  Institute  of  Technology  (MIT) 
Engineering  Systems  Division.  She  is  the 
director  of  MIT  Systems  Engineering 
Advancement  Research  Initiative,  and  also 
affiliated  with  the  MIT  Lean  Advancement 
Initiative.  Dr.  Rhodes  received  her  Doctor  of 
Philosophy  in  Systems  Science,  and  had  20 
years  of  industry  experience  prior  to  joining 
MIT.  She  is  a  past  president  and  Fellow  of  the 
International  Council  on  Systems  Engineering. 

(E-mail  address:  rhodes@mit.edu) 

Col  Larry  Kimm,  USAF,  is  director.  Air  Force 
Human  Systems  Integration  (HSI)  Office, 
Office  of  the  Assistant  Secretary  of  the  Air 
Force  (Acquisition).  He  leads  a  multidisciplinary 
team  developing  Air  Force  and  Department  of 
Defense  HSI  policy,  providing  oversight  of  its 
implementation.  Col  Kimm  held  several 
positions  in  the  Pentagon,  including  Chief, 
Preventive  Medicine,  advising  the  Chairman  of 
the  Joint  Chiefs  of  Staff.  His  last  assignment 
was  at  the  Air  War  College. 

(E-mail  address:  Larry.Klmm@pentagon.af.mil) 

Lt  Col  Alvis  Headen,  USAF,  is  commander, 
Bioenvironmental  Engineer  (BE)  Head¬ 
quarters  at  Air  Combat  Command.  His 
previous  assignment  was  at  the  Air  Force 
Human  Systems  Integration  Office.  He  has 
served  as  BE  flight  commander  at  four  bases 
and  inspected  BE  offices  for  the  Air  Force 
Inspector  General.  Lt  Col  Headen’s  last  two 
operational  assignments  were  at  Seymour- 
Johnson  Air  Force  Base,  North  Carolina,  and 
Dover  Air  Force  Base,  Delaware,  where  he 
commanded  Aeromedical  Dental  Squadrons. 


(E-mall  address:  Alvis. Headen@langley.af  mil) 


3  0  0 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


REFERENCES 

Aronstein,  D.  C.,  Hirschberg,  M.  J.,  &  Piccirillo,  A.  C.  (1998).  Advanced  tactical  fighter  to  F-22 
Raptor:  Origins  of  the  21st  century  air  dominance  fighter.  Reston,  VA:  American  Institute 
of  Aeronautics  and  Astronautics. 

Bolkcom,  C.  (2007).  CRS  report  for  Congress:  F-22A  Raptor.  Washington,  DC:  Congressional 
Research  Service. 

Booher,  H.  R.  (1997).  Human  factors  integration:  Cost  of  and  performance  benefits  to  Army 

systems.  Army  Research  Laboratory  Report  ARL-CR341.  Aberdeen  Proving  Ground,  MD: 
Army  Research  Laboratory. 

Department  of  Defense.  (2008).  Operation  of  the  defense  acquisition  system.  Department  of 
Defense  Instruction  (DoDI)  5000.02.  Washington,  DC:  Government  Printing  Office. 

Department  of  the  Air  Force.  (2008a).  Communications  plan  for  Air  Force  human  systems 
integration.  Falls  Church,  VA:  Air  Force  Human  Systems  Integration  Office. 

Department  of  the  Air  Force.  (2008b).  Fact  sheet:  F-22  Raptor.  Retrieved  December  2008 
from  http://www.af.mil/information/factsheets/factsheet.asp7id  =199 

Department  of  the  Army.  (2005).  MANPRINT handbook.  Washington,  DC:  Office  of  the  Deputy 
Chief  of  Staff  Gl,  MANPRINT  Directorate. 

Department  of  the  Navy.  (2008).  Implementation  and  operation  of  the  defense  acquisition 
system  and  the  joint  capabilities  integration  and  development  system.  Secretary  of  the 
Navy  (SECNAV)  Instruction  5000. 2D.  Washington,  DC:  Government  Printing  Office. 

Deskin,  W.  J.,  &  Yankel,  J.  J.  (2002,  July).  Development  of  the  F-22  propulsion  system. 

Presentation  at  the  38th  AIAA/ASME/SAE/ASEE  Joint  Propulsion  Conference  and 
Exhibit,  Indianapolis,  IN. 

Drew,  C.  (2008,  December  9).  A  fighter  jet’s  fate  poses  a  quandary  for  Obama.  New 

York  Times,  p.  B1.  Retrieved  2009  from  http://www.nytimes.eom/2008/12/10/us/ 
politics/1  Ojets.html 

General  Accounting  Office.  (2003).  Best  practices:  Setting  requirements  differently  could 
reduce  weapon  systems'  total  ownership  costs.  GAO-03-057.  Washington,  DC: 
Government  Printing  Office. 

Gillette,  F.  C.,  Jr.  (1994).  Engine  design  for  mechanics.  SAE International. 

Gunston,  B.  (Ed.)  (2007).  Jane’s  aero-engines.  Alexandria,  VA:  Jane’s  Information  Group. 

Harrison,  J.  A.,  &  Forster,  M.  J.  (2003).  Human  systems  integration  requirements  in  systems 

acquisition.  In  H.  R.  Booher  (Ed.),  Handbook  of  Human  Systems  Integration.  Hoboken,  NJ: 
Wiley- Interscience. 

Haskins,  C.,  Ed.  (2007).  INCOSE systems  engineering  handbook,  v3.1.  Retrieved  2009  from 
http://www.incose.org/ProductsPubs/incosestore.aspx 

Hasselrot,  A.,  &  Montgomerie,  B.  (2005).  An  overview  of  propulsion  systems  for  flying  vehicles. 
Stockholm,  Sweden:  Swedish  Defense  Research  Agency. 

Keith,  D.  R.,  Williams,  J.  G.,  Jr.,  Mullins,  J.  P.,  &  Marsh,  R.  T.  (1983).  Joint  agreement  on  increased 
R&D  for  readiness  and  support.  Alexandria,  VA,  Washington,  DC,  Wright-Patterson  AFB, 
OH:  Department  of  the  Army,  Department  of  the  Navy,  Department  of  the  Air  Force. 

Landsburg,  A.  C.,  Avery,  L.,  Beaton,  R.,  Bost,  J.  R.,  Comperatore,  C.,  Khandpur,  R.,  et  al.  (2008). 
The  art  of  successfully  applying  human  systems  integration.  Nava!  Engineers  Journal, 
720(1),  77-107. 

Liu,  K.  K.,  Valerdi,  R.,  &  Rhodes,  D.H.  (2009,  April).  Economics  of  human  systems  integration: 

A  systems  engineering  perspective.  Proceedings  of  Conference  on  Systems  Engineering 
Research,  Loughborough  University,  United  Kingdom. 

Mack,  D.  D.,  Higgins,  L.  A.,  &  Shattuck,  L.  G.  (2007).  Applying  human  systems  integration  to  the 
rapid  acquisition  process.  Naval  Engineers  Journal,  779(1),  97-108. 

Reynolds,  J.  C.  (1983).  JAFE  field  visit.  Air  Force  memorandum.  Wright-Patterson  AFB,  OH: 


Author. 


The  F119  Engine:  A  Success  Story  of 
Human  Systems  Integration  in  Acquisition 


Valerdi,  R.,  &  Liu,  K.  K.  (2009).  Parametric  cost  estimation  for  human  systems  integration.  In  W. 
B.  Rouse  (Ed.),  The  Economics  of  Human  Systems  Integration.  Manuscript  submitted  for 
publication. 

Wallace,  D.  F.,  Bost,  J.  R.,  Thurber,  J.  B..  &  Hamburger,  P.S.  (2007).  Importance  of  addressing 
human  systems  integration  issues  early  in  the  science  &  technology  process.  Nava! 
Engineers  Journal,  779(1),  59-64. 

Yin,  R.  K.  (2003).  Case  study  research:  Design  and  methods  (3rd  ed.).  Thousand  Oaks,  CA: 
Sage  Publications. 


April  2010  I  3  0  1 


CALL  FOR  AUTHORS 


We  are  currently  soliciting  articles  and  subject  matter 
experts  for  the  2010—2011  Defense  Acquisition  Review 
Journai  (ARJ)  print  years.  Please  see  our  guidelines  for 
contributors  for  submission  deadlines. 

Even  if  your  agency  does  not  require  you  to  publish, 
consider  these  career-enhancing  possibilities: 

•  Share  your  acquisition  research  results  with  the 
acquisition,  technology,  and  Logistics  (AT&L) 
community 

•  Change  the  way  Department  of  Defense(DoD) 
does  business. 

•  Help  others  avoid  pitfalls  with  lessons  learned  or 
best  practices  from  your  project  or  program. 

•  Teach  others  with  a  step-by-step  tutorial  on  a 
process  or  approach. 

•  Share  new  information  that  your  program  has 
uncovered  or  discovered  through  the 
implementation  of  new  initiatives. 

•  Condense  your  graduate  project  into  something 
beneficial  to  acquisition  professionals. 


ENJOY  THESE  BENEFITS 


•  Earn  25  continuous  learning  points  for  publishing  in  a 
refereed  journal. 

•  Get  promoted  or  rewarded. 

•  Become  part  of  a  focus  group  sharing  similar 
interests. 

•  Become  a  nationally  recognized  expert  in 
your  field  or  speciality. 

•  Be  asked  to  speak  at  a  conference  or  symposium. 


A  Publication  of  the  Defense  Acquisition  University 


We  welcome  submissions  from  anyone  involved  in  the 
defense  acquisition  process— the  conceptualization,  initiation, 
design,  testing,  contracting,  production,  deployment,  logistics 
support,  modification,  and  disposal  of  weapons  and  other 
systems,  supplies,  or  services  (including  construction)  needed 
by  the  DoD,  or  intended  for  use  to  support  military  missions. 


If  you  are  interested,  contact  the  Defense  ARJ  managing  editor  (Defen- 
seARJ@dau.mil)  and  provide  contact  information  and  a  brief  descrip¬ 
tion  of  your  article.  Please  visit  the  guidelines  for  authors  at  http:// 
www.dau.mil/pubs/arq/arqart.asp. 


3  04 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


DEFENSE  ARJ 

GUIDELINES  FOR 
CONTRIBUTORS 


The  Defense  Acquisition  Review  Journal  (ARJ)  is  a  scholarly  peer-reviewed 
journal  published  by  the  Defense  Acquisition  University  (DAU).  All  submissions 
receive  a  blind  review  to  ensure  impartial  evaluation. 

IN  GENERAL 

We  welcome  submissions  from  anyone  involved  in  the  defense  acquisition 
process.  Defense  acquisition  is  defined  as  the  conceptualization,  initiation,  design, 
development,  testing,  contracting,  production,  deployment,  logistics  support, 
modification,  and  disposal  of  weapons  and  other  systems,  supplies,  or  services 
needed  by  the  Department  of  Defense  (DoD),  or  intended  for  use  to  support 
military  missions. 

We  encourage  prospective  writers  to  coauthor,  adding  depth  to  manuscripts. 
It  is  recommended  that  a  mentor  be  selected  who  has  been  previously  published 
or  has  expertise  in  the  manuscript’s  subject. 

Authors  should  be  familiar  with  the  style  and  format  of  previous  Defense  ARJs 
and  adhere  to  the  use  of  endnotes  versus  footnotes,  formatting  of  reference  lists  or 
bibliographies,  and  the  use  of  designated  style  guides,  it  is  aiso  the  responsibility 
of  the  corresponding  author  to  furnish  a  government  agency/employer  clearance 
with  each  submission. 


MANUSCRIPTS 

Manuscripts  should  reflect  research  of  empirically  supported  experience  in 
one  or  more  of  the  areas  of  acquisition  discussed  above.  Research,  lessons  learned, 
or  tutorial  articles  should  not  exceed  4,500  words.  Opinion  articles  should  be 
limited  to  1,500  words. 

Research  articles  are  characterized  by  a  systematic  inquiry  into  a  subject 
to  discover/revise  facts  or  theories.  Tutorials  should  provide  special  instruction 
or  knowledge  relevant  to  an  area  of  defense  acquisition  and  be  of  benefit  to  the 
Defense  Acquisition  Workforce.  Opinion  articles  should  reflect  judgments  based 
on  the  special  knowledge  of  the  expert  and  should  be  based  on  observable 
phenomena  and  presented  in  a  factual  manner,  implying  detachment. 


Defense  ARJ  Guidelines  for  Contributors 


April  2010 


3  0  5 


STYLE 

We  will  require  you  to  submit  your  final  draft  of  the  manuscript,  especially 
citations  (endnotes  instead  of  footnotes),  in  the  format  specified  in  two  specific 
style  manuals.  The  ARJ  follows  the  author  (date)  form  of  citation.  We  expect  you 
to  use  the  Publication  Manual  of  the  American  Psychological  Association  (6th 
Edition)  and  the  Chicago  Manual  of  Style  (15th  Edition). 

Contributors  are  encouraged  to  seek  the  advice  of  a  reference  librarian  in 
completing  citation  of  government  documents  because  standard  formulas  of 
citations  may  provide  incomplete  information  in  reference  to  government  works. 
Helpful  guidance  is  also  available  in  Garner,  D.  L.,  and  Smith,  D.  H.,  1993,  The 
Complete  Guide  to  Citing  Government  Documents:  A  Manual  for  Writers  and 
Librarians  (Rev.  Ed.),  Bethesda,  MD:  Congressional  Information  Service. 


FORMAT 

Pages  should  be  double-spaced  and  organized  in  the  following  order:  title 
page,  abstract  (120  words  or  less),  list  of  keywords,  body,  reference  list,  author’s 
note  (if  any),  and  any  figures  or  tables.  Figures  or  tables  should  not  be  inserted  (or 
embedded,  etc.)  into  the  text,  but  segregated  (one  to  a  page)  following  the  text. 
When  material  is  submitted  electronically,  each  figure  or  table  should  be  saved 
to  a  separate,  exportable  file  (i.e.,  a  readable  EPS  file).  For  additional  information 
on  the  preparation  of  figures  or  tables,  see  CBE  Scientific  Illustration  Committee, 
1988,  Illustrating  Science:  Standards  for  Publication,  Bethesda,  MD:  Council  of 
Biology  Editors.  Restructure  briefing  charts  and  slides  to  look  similar  to  those  in 
previous  issues  of  the  ARJ. 

The  author  (or  corresponding  author  in  cases  of  multiple  authors)  should 
attach  to  the  manuscript  a  signed  cover  letter  that  provides  all  of  the  authors' 
names,  mailing  and  e-mail  addresses,  as  well  as  telephone  and  fax  numbers.  The 
letter  should  verify  that  the  submission  is  an  original  product  of  the  author;  that 
it  has  not  been  previously  published;  and  that  it  is  not  under  consideration  by 
another  publication.  Details  about  the  manuscript  should  also  be  included  in  this 
letter;  for  example,  title,  word  length,  a  description  of  the  computer  application 
programs,  and  file  names  used  on  enclosed  CDs,  e-mail  attachments,  or  other 
electronic  media. 


COPYRIGHT 

The  ARJ  is  a  publication  of  the  United  States  Government  and  as  such  is  not 
copyrighted.  Because  the  ARJ  is  posted  as  a  complete  document  on  the  DAU 
homepage,  we  will  not  accept  copyrighted  articles  that  require  special  posting 
requirements  or  restrictions.  If  we  do  publish  your  copyrighted  article,  we  will  print 
only  the  usual  caveats.  The  work  of  federal  employees  undertaken  as  part  of  their 
official  duties  is  not  subject  to  copyright  except  in  rare  cases. 

In  citing  the  work  of  others,  it  is  the  contributor's  responsibility  to  obtain 
permission  from  a  copyright  holder  if  the  proposed  use  exceeds  the  fair  use 


3  0  6 


A  Publication  of  the  Defense  Acquisition  University 


www.dau.mil 


provisions  of  the  law  (see  U.S.  Government  Printing  Office,  1994,  Circular  92: 
Copyright  Law  of  the  United  States  of  America,  p.  15,  Washington,  DC:  Author). 
Contributors  will  be  required  to  submit  a  copy  of  the  writer’s  permission  to  the 
Managing  Editor  before  publication. 


POLICY 

We  reserve  the  right  to  decline  any  article  that  fails  to  meet  the  following 
copyright  requirements: 

•  The  author  cannot  obtain  official  permission  to  use  previously 
copyrighted  material  in  the  article. 

•  The  author  will  not  allow  DAU  to  post  the  article  in  our  ARJ  issue 
on  our  Internet  homepage. 

•  The  author  requires  that  usual  copyright  notices  be  posted  with 
the  article. 

•  To  publish  the  article  requires  copyright  payment  by  the  DAU 
press. 


SUBMISSION 

All  manuscript  submissions  should  include  the  following: 

•  Cover  letter 

•  Biographical  sketch  for  each  author 

•  Headshot  for  each  author  (saved  to  a  CDR  disk  as  a  300  dots 
per  inch  [dpi]  or  high-print  quality  JPEG  or  Tiff  file.  Please  note: 
images  from  web,  PowerPoint  or  e-mail  will  not  be  accepted  due 
to  low  image  quality.) 

•  One  copy  of  the  typed  manuscript 

These  items  should  be  sent  electronically,  as  appropriately  labeled  files,  to 
Defense  ARJ  Managing  Editor,  Norene  Fagan-Blanch  at:  Norene.Fagan-Blanch@ 
dau.mil. 

In  addition  to  electronic  submissions,  a  hard  copy  of  manuscript  and  any 
diskettes  should  be  mailed  to:  Department  of  Defense,  Defense  Acquisition 
University,  Attn:  DAU  Press  (Defense  A/t’U  Managing  Editor),  Suite  3,  9820  Belvoir 
Road,  Fort  Belvoir,  VA  22060-5565. 


Defense  ARJ  Guidelines  for  Contributors 


April  2010 


3  0  7 


DEFENSE  ARJ 

PRINT  SCHEDULE 


The  Defense  ARJ  is  published  in  quarterly  theme  editions.  Please  consult 
the  DAU  home  page  for  current  themes  being  solicited.  See  print  schedule 
below. 


2010 


Publication  Date 

January  2010 
April  2010 
July  2010 
October  2010 


Due  Date 

July  1.  2009 
November  16,  2009 
January  4, 2010 
April  2,  2010 


Publication  Date 

January  2011 
April  2011 
July  2011 
October  2011 


Due  Date 

July  1.  2010 
November  1,  2010 
January  3,  2011 
April  1,  2011 


In  most  cases,  the  author  will  be  notified  that  the  submission  has 
been  received  within  48  hours  of  its  arrival.  Following  an  initial  review, 
submissions  will  be  referred  to  referees  and  for  subsequent  consideration 
by  the  Executive  Editor,  Defense  ARJ. 

Contributors  may  direct  their  questions  to  the  Managing  Editor,  Defense 
ARJ  by  calling  703-805-3801  (fax:  703-805-2917),  or  via  the  Internet  at 
norene.fagan-blanch@dau.mil.  The  DAU  Home  Page  can  be  accessed  at: 
http://www.dau.mil. 


DEFENSE  ACQUISITION  UNIVERSITY 

WEB  SITE 


http://www.dau.mn 


Your  Online  Access  to  Acquisition  Research,  Consulting, 
Information,  and  Course  Offerings 


DAU  HOME  PAGE 

•  Performance  Learning  Model 

•  Acquire  (Search) 

•  DAU  Online  Resources 

I  NEED  TRAINING 

•  General  Information 

•  Course  Schedules 

•  Apply  for  a  Course 

•  Pre-Course  Information 

•  Login— Distance  Learning  Courses 

•  Browse  Distance  Learning  Courses 

CONTINUOUS  LEARNING 

•  Register  for  Continuous  Learning 
Modules 

•  Browse  Continuous  Learning  Modules 

•  Login— Continuous  Learning  Modules 

•  Conferences  and  Symposiums 

•  General  Information 


KNOWLEDGE  SHARING 

•  Defense  Acquisition  Portal  (DAP) 

•  Defense  Acquisition  Policy  Center 

•  Acquisition  Community  Connection  (ACC) 

•  Acquisition  Best  Practices 
Clearinghouse  (BPCh) 

•  ACQuIpcdia 

•  Defense  Acquisition  Guidebook  (DAG) 

•  Integrated  Framework  Chart  (IFC) 

•  Program  Managers  e-Tool  Kit 

•  DAU  Virtual  Library 

MISSION  ASSISTANCE 

•  Description  and  Contacts 

•  Consulting 

‘  Facilitated  Decision  Making 

•  Rapid  Deployment  Training 

•  Targeted  Training 

•  Ask  a  Professor  (AAP) 


Now  you  can  search  the  DAU  Web  site  and  our  online  publications! 


First  initial 
of  last  name 


SUBSCRIPTION 


Defense  ARJ  1  1 

Ouantitv 

Defense  AT&L  1  1 

Ouantitv 

1  1  New  Subscription 

1  1  Cancellation 

1  1  Change  of  Address 

Date  Form  Completed 
Last  Name 
First  Name 
New  Address 


Old  Address 


DayAVork  Phone 
E-Mail  Address  (Optional) 


Signature  (Required) 


PLEASE  MAIL  OR  LAX  TO;  703-805-2917 
The  Privacy  Act  and  Ereedom  of  Information  Act 

If  you  provide  us  your  business  address,  you  may  become  part  of  mailing  lists  we  are  required 
to  provide  to  other  agencies  who  request  the  lists  as  public  information. 

If  you  prefer  not  to  be  part  of  these  lists,  use  your  home  address.  Please  do  not  include  your 
rank,  grade,  service,  or  other  personal  identifiers. 


NO  POSTAGE 
NECESSARY 
IF  MAILED 
IN  THE 

UNITED  STATES 


BUSINESS  REPLY  MAIL 

FIRST-CLASS  MAIL  PERMIT  NO.  12  FORT  BELVOIR.VA 


POSTAGE  WILL  BE  PAID  BY  ADDRESSEE 


DEFENSE  ACQUISITION  UNIVERSITY 

ATTN:  DAU  PRESS 
9820  BELVOIR  RD  STE  3 
FORT  BELVOIR  VA  22060-9910 


ImI.ImI.IIImmIImII.mI.ImI.I . I 


A  publication  of  the  Defense  Acquisition  University 


The  Defense  Acquisition  Review  Journal 
^  w7re  on  the  web  at:  http;//twww.dau.mil/pubscats/Pages/ARJ.asp« 


Artkl**  rapfCMnt  the  views  of  the  euthors  end  da  not  necesserlly  reflect  the  opinion  of  DAU  or  the  Department  of  Defense. 


A  Publication  of  the  Defense  Acquisition  University 

Learn.  Perform.  Succeed. 


